Competency Mining in Large Data Sets - Preparing Large Scale Investigations in Computer Science Education

Peter Hubwieser, Andreas Mühling

2014

Abstract

In preparation of large scale surveys on computer science competencies, we are developing proper competency models and evaluation methodologies, aiming to define competencies by sets of exiting questions that are testing congruent abilities. For this purpose, we have to look for sets of test questions that are measuring joint psychometric constructs (competencies) according to the responses of the test persons. We have developed a methodology for this goal by applying latent trait analysis on all combinations of questions of a certain test. After identifying suitable sets of questions, we test the fit of the mono-parametric Rasch Model and evaluate the distribution of person parameters. As a test bed for first feasibility studies, we have utilized the large scale Bebras Contest in Germany 2009. The results show that this methodology works and might result one day in a set of empirically founded competencies in the field of Computational Thinking.

References

  1. Andersen, EB 1973, 'A goodness of fit test for the rasch model', Psychometrika, vol. 38, no. 1, pp. 123-140.
  2. Bamberg, G, Baur, F & Krapp, M 2011, Statistik, Oldenbourg, München.
  3. Bartholomew, DJ, Steel, F, Moustaki, I & Galbrath, JI 2008, Analysis of multivariate social science data, CRC Press / Taylor & Francis, Boca Raton Fla.u.a.
  4. Dagiene, V 2008, 'Bebras International Contest on Informatics and Computer Literacy Criteria for Good Tasks', Lecture notes in computer science, pp. No. 5090 (2008),19-30.
  5. Dagiene, V & Futschek, G 2008, 'Bebras International Contest on Informatics and Computer Literacy: Criteria for Good Tasks'. Informatics Education - Supporting Computational Thinking, Third International Conference on Informatics in Secondary Schools - Evolution and Perspectives, ISSEP 2008, Torun, Poland, July 1-4, 2008, eds RT Mittermeir & MM Syslo, Springer, pp. 19-30.
  6. Klieme, E, Avenarius, H, Blum, W, Döbrich, P, Gruber, H, Prenzel, M, Reiss, K, Riquarts, K, Rost, J, Tenorth, H & Vollmer, HJ 2004, The Development of National Educational Standards. An Expertise, Bundesministerium für BildungundForschung, Berlin.
  7. Magenheim, J, Nelles, W, Rhode, T, Schaper, N, Schubert, SE & Stechert, P 2010, 'Competencies for informatics systems and modeling: Results of qualitative content analysis of expert interviews'. Education Engineering (EDUCON), 2010 IEEE, pp. 513-521.
  8. Martin-Löf, P 1974, 'Exact tests, confidence regions and estimates' in Proceedings of Conference on Foundational Questions in Statistical Inference. (Aarhus, 1973), Univ. Aarhus, Aarhus, pp. 121-138.
  9. OECD (ed.) 2013, PISA 2012 Results in Focus. What 15- year-olds know and what they can do with what they know, OECD Publishing, Paris.
  10. Rizopoulos, D 2006, 'ltm: An R Package for Latent Variable Modeling and Item Response Analysis', Journal of Statistical Software, vol. 17, no. 5, pp. 1-25.
  11. Rost, J & Carstensen, CH 2002, 'Multidimensional Rasch Measurement via Item Component Models and Faceted Designs', Applied Psychological Measurement, vol. 26, no. 1, pp. 42-56.
  12. Schott, F & Azizi Ghanbari, S 2009, 'Modellierung, Vermittlung und Diagnostik der Kompetenz kompetenzorientiert zu unterrichten - wissenschaftliche Herausforderung und ein praktischer Lösungsversuch', Lehrerbildung auf dem Prüfstand, vol. 2, no. 1, pp. 10-27.
  13. Seidel, T & Prenzel, M 2008, 'Assessment in Large-Scale Studies' in Assessment of competencies in educational contexts, eds E Klieme, D Leutner & J Hartig, Hogrefe & Huber Publishers, Toronto, pp. 279-304.
  14. Tucker, A, Seehorn, D, Carey, S, Moix, D, Fuschetto, B, Lee, I, O'Grady-Cuniff, D, Stephenson, C & Verno, A 2011, CSTA K-12 Computer Science Standards. Revised 2011. CSTA Standards Task Force, CSTA. Available from:
  15. Wald, A 1943, 'Tests of Statistical Hypotheses Concerning Several Parameters When the Number of Observations is Large', Transactions of the American Mathematical Society 1943, pp. 426-482.
  16. Weinert, FE 2001, 'Concept of Competence: A conceptual clarification.78 in Defining and Selecting Key Competencies, eds DS Rychen & L Salganik, Hogrefe and Huber, Seattle.
  17. Wing, JM 2006, 'Computational thinking', Commun. ACM, vol. 49, no. 3, pp. 33-35.
Download


Paper Citation


in Harvard Style

Hubwieser P. and Mühling A. (2014). Competency Mining in Large Data Sets - Preparing Large Scale Investigations in Computer Science Education . In Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2014) ISBN 978-989-758-048-2, pages 315-322. DOI: 10.5220/0005129203150322


in Bibtex Style

@conference{kdir14,
author={Peter Hubwieser and Andreas Mühling},
title={Competency Mining in Large Data Sets - Preparing Large Scale Investigations in Computer Science Education},
booktitle={Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2014)},
year={2014},
pages={315-322},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005129203150322},
isbn={978-989-758-048-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: KDIR, (IC3K 2014)
TI - Competency Mining in Large Data Sets - Preparing Large Scale Investigations in Computer Science Education
SN - 978-989-758-048-2
AU - Hubwieser P.
AU - Mühling A.
PY - 2014
SP - 315
EP - 322
DO - 10.5220/0005129203150322