![](bg9.png)
Table 5: Mean and Stander Deviation of survey questions for evaluating auto-generated tutoring dialogues by instructors.
Question Mean SD
I think the generated questions, feedback, and answers are syntactically correct. 4.2 0.55
I think the generated questions, feedback, and answers are semantically correct. 4.5 0.84
I think the generated dialogue is coherent and consistent. 4.4 0.45
I think the generated dialogue (questions, feedback) would help students understand Java
code.
4.2 0.84
I think the generated dialogue (questions, feedback) would help students develop a robust
understanding of programming concepts.
3.8 0.55
I think the generated scaffolding questions would help students learn and understand the
corresponding Java code.
4.6 0.55
I think the generated dialogue covers all important programming concepts presented in
the code.
4.8 0.45
I think I may use this system in the classroom. 4.2 0.84
I think the system is effective at helping students understand Java code. 4.2 0.45
I think the system is effective at helping students understand programming concepts. 4.2 0.84
ity. The result shows that the teachers believe the di-
alogues as good. However, the instructors believed
that more improvement is required to help students
developing a robust understanding of programming
concepts.
ACKNOWLEDGEMENTS
This work as partially funded by the National Science
Foundation under the award #1822816 (Collaborative
Research: CSEdPad: Investigating and Scaffolding
Students’ Mental Models during Computer Program-
ming Tasks to Improve Learning, Engagement, and
Retention) to Dr. Vasile Rus. All opinions stated or
implied are solely the authors’ and do not reflect the
opinions of the funding agency.
REFERENCES
Ainsworth, S., Major, N., Grimshaw, S., Hayes, M., Un-
derwood, J., Williams, B., and Wood, D. (2003). Re-
deem: Simple intelligent tutoring systems from usable
tools. In Authoring Tools for Advanced Technology
Learning Environments, pages 205–232. Springer.
Aleven, V., Mclaren, B. M., Sewall, J., and Koedinger, K. R.
(2009). A new paradigm for intelligent tutoring sys-
tems: Example-tracing tutors. International Journal
of Artificial Intelligence in Education, 19(2):105–154.
Alshaikh, Z., Tamang, L. J., and Rus, V. (2020). Exper-
iments with a socratic intelligent tutoring system for
source code understanding. In The Thirty-Third Inter-
national Florida Artificial Intelligence Research Soci-
ety Conference (FLAIRS-32).
Aroyo, L., Inaba, A., Soldatova, L., and Mizoguchi, R.
(2004). Ease: Evolutional authoring support environ-
ment. In International Conference on Intelligent Tu-
toring Systems, pages 140–149. Springer.
Blessing, S. B. (1997). A programming by demonstra-
tion authoring tool for model-tracing tutors. Interna-
tional Journal of Artificial Intelligence in Education
(IJAIED), 8:233–261.
Bloom, B. S. (1984). The 2 sigma problem: The search
for methods of group instruction as effective as one-
to-one tutoring. Educational researcher, 13(6):4–16.
Cohen, P. A., Kulik, J. A., and Kulik, C.-L. C. (1982).
Educational outcomes of tutoring: A meta-analysis
of findings. American educational research journal,
19(2):237–248.
Corbett, A., Anderson, J., Graesser, A., Koedinger, K., and
VanLehn, K. (1999). Third generation computer tu-
tors: learn from or ignore human tutors? In CHI’99
Extended Abstracts on Human Factors in Computing
Systems, pages 85–86, New York, NY, USA. Associa-
tion for Computing Machinery.
Freedman, R., Ros
´
e, C. P., Ringenberg, M. A., and Van-
Lehn, K. (2000). Its tools for natural language dia-
logue: A domain-independent parser and planner. In
International Conference on Intelligent Tutoring Sys-
tems, pages 433–442. Springer.
Graesser, A. C., D’Mello, S., and Person, N. (2009). 19
meta-knowledge in tutoring. Handbook of metacogni-
tion in education, page 361.
Halff, H. M., Hsieh, P. Y., Wenzel, B. M., Chudanov, T. J.,
Dirnberger, M. T., Gibson, E. G., and Redfield, C. L.
(2003). Requiem for a development system: reflec-
tions on knowledge-based, generative instruction. In
Authoring tools for advanced technology learning en-
vironments, pages 33–59. Springer.
Heffernan, N. T., Turner, T. E., Lourenco, A. L., Macasek,
M. A., Nuzzo-Jones, G., and Koedinger, K. R. (2006).
The assistment builder: Towards an analysis of cost
effectiveness of its creation. In Flairs Conference,
pages 515–520.
Jordan, P., Ros
´
e, C. P., and VanLehn, K. (2001). Tools for
authoring tutorial dialogue knowledge. In Proceed-
ings of AI in Education 2001 Conference.
Experiments with Auto-generated Socratic Dialogue for Source Code Understanding
43