SYSTEMATIC GENERATION IN DCR EVALUATION PARADIGM - Application to the Prototype CLIPS system

Mohamed Ahafhaf

2005

Abstract

In this paper we present an extension of DCR evaluation method tested on a spoken language understanding and dialog system. It should allow a deep evaluation of spoken language understanding and dialog systems. The key point of our method is the use of a linguistic typology in order to generate an evaluation corpus that covers a significant number of the linguistic phenomena we want to evaluate our system on. This allows having a more objective and deep evaluation of spoken language understanding and dialog systems.

References

  1. Andrews, A., 1985, The major functions of the noun phrase, In T. SHOPEN (editor), Language typology and syntactic description, Vol. 1 Cambridge university press.
  2. Antoine, J-Y., Siroux, J., Caelen, J., Villaneau, J., Goulian, J., Ahafhaf, M., 2000, Obtaining predictive results with an objective evaluation of spoken dialogue systems: experiments with the DCR assessment paradigm, LREC'2000, Athens, Greece.
  3. Benveniste, C-B.,1990, le français parlé : études grammaticales, Editions du CNRS, Paris.
  4. Benveniste, C-B., 1997, Approches de la langue parlée en français, Ophrys, Paris.
  5. Blasco-Dulbecco, M., 1999, Les dislocations en français contemporain : étude syntaxique, Honoré Champion, Paris.
  6. Caelen, J., Stratégies de dialogue, Conférence MFI'(Modèles Formels de l'Interaction), Lille, Cépadues éd, 2003.
  7. Gadet, F., 1992, Le français populaire, Paris : Armand Colin, 1992.
  8. Kurdi, M.Z., 2001, A spoken language understanding approach which combines the parsing robustness with the interpretation deepness, proceedings of the International Conference on Artificial Intelligence ICAI01, Las Vegas, USA, June 25 - 28.
  9. Kurdi, M.Z., Ahafhaf, M., 2003, A grammar based method for systematic and generic spoken language understanding systems evaluation, In International Conference on Natural Language Processing and Knowledge Engineering, Beijing Media Center, China, October 26-29.
  10. Nguyen H., Caelen J. 2003, Generic manager for spoken dialogue systems. DiaBruck : 7th Workshop on the Semantics and Pragmatics of Dialogue, Proceedings pp.201-202.
  11. Picabia, L., 1975, Eléments de grammaire générative : application au français, Armand Colin, Paris.
  12. Riegel, M., Pellat, J-C., Rioul, R., 1994, Grammaire méthodique du français, PUF, Paris.
  13. Sabah, G., 1997, Rapport final du projet DALI (Dialogue Adaptatif: Langue et Interaction),
  14. Zeiliger, J., Caelen, J., Antoine, J.Y, 1997, Vers une méthodologie d'évaluation qualitative des systèmes de compréhension et de dialogue oral homme-machine, actes JSTFRANCIL' 97, Avignon, France,4.
Download


Paper Citation


in Harvard Style

Ahafhaf M. (2005). SYSTEMATIC GENERATION IN DCR EVALUATION PARADIGM - Application to the Prototype CLIPS system . In Proceedings of the Seventh International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 972-8865-19-8, pages 374-380. DOI: 10.5220/0002520603740380


in Bibtex Style

@conference{iceis05,
author={Mohamed Ahafhaf},
title={SYSTEMATIC GENERATION IN DCR EVALUATION PARADIGM - Application to the Prototype CLIPS system},
booktitle={Proceedings of the Seventh International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2005},
pages={374-380},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002520603740380},
isbn={972-8865-19-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Seventh International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - SYSTEMATIC GENERATION IN DCR EVALUATION PARADIGM - Application to the Prototype CLIPS system
SN - 972-8865-19-8
AU - Ahafhaf M.
PY - 2005
SP - 374
EP - 380
DO - 10.5220/0002520603740380