Item Difficulty Analysis of English Vocabulary Questions

Yuni Susanti, Hitoshi Nishikawa, Takenobu Tokunaga, Obari Hiroyuki

2016

Abstract

This study investigates the relations between several factors of question items in English vocabulary tests and the corresponding item difficulty. Designing the item difficulty of a test impacts the quality of the test itself. Our goal is suggesting a way to control the item difficulty of questions generated by computers. To achieve this goal we conducted correlation and regression analyses on several potential factors of question items and their item difficulty obtained through experiments. The analyses revealed that several item factors correlated with the item difficulty, and up to 59% of the item difficulty can be explained by a combination of item factors.

References

  1. Bachman, L. F. (1990). Fundamental Consideration in Language Testing. Oxford University Press.
  2. Beinborn, L., Zesch, T., and Gurevych, I. (2014). Predicting the difficulty of language proficiency tests. In Transactions of the Association for Computational Linguistics, volume 2, pages 517-529. Association for Computational Linguistics.
  3. Brown, J. C., Frishkoff, G. A., and Eskenazi, M. (2005). Automatic question generation for vocabulary assessment. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 819-826.
  4. Brown, J. D. (1989). Cloze item difficulty. In Japan Association for Language Teaching Journal, volume 11, No.1, pages 46-67. JALT.
  5. Brown, J. D. (2012). Classical test theory. In Fulcher, G. and Davidson, F., editors, The Routledge Handbook of Language Testing, chapter 22, pages 323-335. Routledge.
  6. ETS (2007). The Official Guide to the New TOEFL iBT Internation edition. Mc Graw-Hill.
  7. Fellbaum, C. (1998). WordNet: A lexical database for English. A Bradford Book.
  8. Gear, J. and Gear, R. (2006). Cambridge Preparation for the TOEFL Test 4th Edition. Cambridge University Press;.
  9. Heilman, M., Collins-Thompson, K., and Eskenazi, M. (2008). An analysis of statistical models and features for reading difficulty prediction. In Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications, EANL 7808, pages 71- 79, Stroudsburg, PA, USA. Association for Computational Linguistics.
  10. Lee, J. and Seneff, S. (2007). Automatic generation of cloze items for prepositions. In Proceedings of Interspeech 2007, pages 2173-2176.
  11. Lin, Y.-C., Sung, L.-C., and Chen, M. C. (2007). An automatic multiple-choice question generation scheme for English adjective understanding. In Proceedings of Workshop on Modeling, Management and Generation of Problems/Questions in eLearning, the 15th International Conference on Computers in Education (ICCE 2007), pages 137-142.
  12. McCarthy, D. (2009). Word sense disambiguation: An overview. Language and Linguistics Compass, 3(2):537-558.
  13. Medero, J. and Ostendorf, M. (2009). Analysis of vocabulary difficulty using wiktionary. In Proceedings of the Speech and Language Technology in Education Workshop (SLaTE).
  14. Petersen, S. E. and Ostendorf, M. (2009). A machine learning approach to reading level assessment. Comput. Speech Lang., 23(1):89-106.
  15. Phillips, D. (2006). Longman Preparation Course for the TOEFL Test: iBT. Pearson Education Inc.
  16. Sakaguchi, K., Arase, Y., and Komachi, M. (2013). Discriminative approach to fill-in-the-blank quiz generation for language learners. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistic, pages 238-242. Association for Computational Linguistic.
  17. Sharpe, P. J. (2006). Barron's TOEFL iBT Internet-Based Test 2006-2007 12th Edition with CD-ROM. Barron's Educational Series Inc.
  18. Sigott, G. (1995). The c-test: some factors of difficulty. In AAA: Arbeiten aus Anglistik und Amerikanistik, 20(1), volume 20(1), pages 43-53. Narr Francke Attempto Verlag GmbH Co. KG.
  19. Susanti, Y., Iida, R., and Tokunaga, T. (2015). Automatic generation of english vocabulary tests. In Proceedings of the 7th International Conference on Computer Supported Education, pages 77-87.
  20. Uemura, T. and Ishikawa, S. (2004). JACET 8000 and asia TEFL vocabulary initiative. In Journal of ASIA TEFL, volume 1(1), pages 333-347. ASIA TEFL).
Download


Paper Citation


in Harvard Style

Susanti Y., Nishikawa H., Tokunaga T. and Hiroyuki O. (2016). Item Difficulty Analysis of English Vocabulary Questions . In Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-179-3, pages 267-274. DOI: 10.5220/0005775502670274


in Bibtex Style

@conference{csedu16,
author={Yuni Susanti and Hitoshi Nishikawa and Takenobu Tokunaga and Obari Hiroyuki},
title={Item Difficulty Analysis of English Vocabulary Questions},
booktitle={Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2016},
pages={267-274},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005775502670274},
isbn={978-989-758-179-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Item Difficulty Analysis of English Vocabulary Questions
SN - 978-989-758-179-3
AU - Susanti Y.
AU - Nishikawa H.
AU - Tokunaga T.
AU - Hiroyuki O.
PY - 2016
SP - 267
EP - 274
DO - 10.5220/0005775502670274