Table 1: Performance of every classifier with 10-fold cross validation. Accuracy (κ).
Affect Na
¨
ıve Bayes J-48 k-NN ANN
Flow/engaged 79.00% (0.50) 80.48% (0.51) 76.14% (0.43) 78.43% (0.45)
Relaxation 71.10% (0.40) 72.57% (0.45) 69.14% (0.37) 71.24% (0.34)
Distraction 70.33% (0.43) 71.00% (0.43) 74.00% (0.50) 72.52% (0.39)
Frustration 74.71% (0.49) 73.86% (0.45) 72.62% (0.42) 78.00% (0.50)
Boredom 74.71% (0.47) 84.57% (0.59) 83.81% (0.58) 76.19% (0.45)
in this work the results obtained were similar to those
works applied to fixed-text dynamics. A possible ex-
planation of this would be the addition of the mouse
dynamics features and the additional preprocessing
performed on the feature vectors. Out of all methods
tested, a multilayer perceptron trained with backprop-
agation, the one tagged ANN in RapidMiner, gave the
best classification results.
While these are promising results further experi-
ments are needed. Other experiments should focus on
novice programmers, but that is left as future work.
ACKNOWLEDGMENT
This work has been supported in part by: de
Ministerio espa
˜
nol de Econom
´
ıa y Competitivi-
dad under project TIN2014-56494-C4-3-P (UGR-
EPHEMECH) and by CONACYT PEI Project No.
220590.
REFERENCES
Bakhtiyari, K. and Husain, H. (2014). Fuzzy model
on human emotions recognition. arXiv preprint
arXiv:1407.1474.
Bangert-Drowns, R. L. and Pyke, C. (2002). Teacher ratings
of student engagement with educational software: An
exploratory study. Educational Technology Research
and Development, 50(2):23–37.
Bennedsen, J. and Caspersen, M. E. (2007). Failure rates
in introductory programming. ACM SIGCSE Bulletin,
39(2):32–36.
Bixler, R. and D´Mello, S. (2013). Detecting boredom and
engagement during writing with keystroke analysis,
task appraisals, and stable traits. In Proceedings of
the 2013 international conference on Intelligent user
interfaces, pages 225–234. ACM.
Blikstein, P. (2011). Using learning analytics to assess
students’ behavior in open-ended programming tasks.
In Proceedings of the 1st international conference on
learning analytics and knowledge, pages 110–116.
ACM.
Bosch, N., D´Mello, S., and Mills, C. (2013). What emo-
tions do novices experience during their first computer
programming learning session? In International Con-
ference on Artificial Intelligence in Education, pages
11–20. Springer.
Busjahn, T., Schulte, C., Sharif, B., Begel, A., Hansen, M.,
Bednarik, R., Orlov, P., Ihantola, P., Shchekotova, G.,
Antropova, M., et al. (2014). Eye tracking in com-
puting education. In Proceedings of the tenth annual
conference on International computing education re-
search, pages 3–10. ACM.
Csikszentmihalyi, M. (1990). Flow: The psychology of op-
timal performance. NY: Cambridge UniversityPress.
Dijkstra, E. W. et al. (1989). On the cruelty of really teach-
ing computing science. Communications of the ACM,
32(12):1398–1404.
Dillenbourg, P., Schneider, D., and Synteta, P. (2002). Vir-
tual learning environments. In 3rd Hellenic Con-
ference” Information & Communication Technolo-
gies in Education”, pages 3–18. Kastaniotis Editions,
Greece.
D´Mello, S., Jackson, T., Craig, S., Morgan, B., Chipman,
P., White, H., Person, N., Kort, B., el Kaliouby, R., Pi-
card, R., et al. (2008). Autotutor detects and responds
to learners affective and cognitive states. In Workshop
on emotional and cognitive issues at the international
conference on intelligent tutoring systems, pages 306–
308.
Elliott, C., Rickel, J., and Lester, J. (1999). Lifelike
pedagogical agents and affective computing: An ex-
ploratory synthesis. In Artificial intelligence today,
pages 195–211. Springer.
Epp, C., Lippold, M., and Mandryk, R. L. (2011). Identi-
fying emotional states using keystroke dynamics. In
Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems, pages 715–724. ACM.
Jenkins, T. (2001). The motivation of students of program-
ming. In ACM SIGCSE Bulletin, volume 33, pages
53–56. ACM.
Jenkins, T. (2002). On the difficulty of learning to program.
In Proceedings of the 3rd Annual Conference of the
LTSN Centre for Information and Computer Sciences,
volume 4, pages 53–58. Citeseer.
Kapoor, A. and Picard, R. W. (2005). Multimodal affect
recognition in learning environments. In Proceedings
of the 13th annual ACM international conference on
Multimedia, pages 677–682. ACM.
Khan, I. A., Hierons, R. M., and Brinkman, W. P. (2007).
Mood independent programming. In Proceedings
of the 14th European conference on Cognitive er-
gonomics: Invent! Explore!, pages 269–272. ACM.
Kołakowska, A. (2013). A review of emotion recogni-
tion methods based on keystroke dynamics and mouse
movements. In 2013 6th International Conference