![](bg2.png)
2 RELATED WORK
For the purpose of this paper, we divide educational
software supporting programming education into two
categories. The first category are intelligent tutoring
systems tailored to support the understanding of fun-
damental programming concepts. The other category
are automated grading tools which are also often used
in the context of programming contests.
Intelligent Tutoring Systems. Tutoring tools for
programming education date back to the early 1980’s
(Johnson and Soloway, 1984; Soloway, 1986). The
goal of a tutoring system is to analyse student re-
sponses and gather information about the learning
process (Brusilovsky, 1995; Corbett and Anderson,
2008). Thus, they are able to assist students, point
out weaknesses and strength and suggest further study
material.
Tutoring systems typically present learning units
in small cognitive accessible exercises or dialogs
(Lane and VanLehn, 2004). They may also coach
students with hints, tips and additional information
throughout the study process (Lane and VanLehn,
2003). This step-by-step process is not always ap-
propriate. Teachers also want students to engage in
more complex problem solving tasks without or with
less guidance. These types of exercises often cannot
be properly covered with tutoring systems.
The progress in research on intelligent tutoring
systems has also been accompanied by the evolution
of new programming concepts and languages. Partic-
ularly, the rise of object oriented programming lan-
guages has changed how programming is taught and
how tutoring systems are implemented (Sykes and
Franek, 2003). While object orientation is certainly
the main focus of today’s programming education,
there are tutors for other popular concepts as e.g.
functional programming (Xu and Sarrafzadeh, 2004).
Modern tutoring systems are often web-based (Butz
et al., 2004) with a central server analyzing source
code, building student models and offering help and
assistance to students.
Automated Grading Systems. The development of
grading tools happened along two different paths.
One motivation for their use were large scale pro-
gramming contests, where students compete in solv-
ing algorithmic problems (Leal and Silva, 2003).
Usually, problems are designed to require complex
program logic, but to produce simple output (e.g. find
the shortest route to escape a maze). Some contest
software also evaluates source code and program met-
rics, e.g. number of lines of the implementation or
program speed (Leal and Silva, 2008).
In recent years, grading tools have also become
more popular as learning support tools, allowing
teachers to spend more time working with students
and less time grading course assignments (Tiantian
et al., 2009; Hukk et al., 2011; Foxley et al., 2001).
Grading tools usually evaluate the correctness of
a complete computer program. That is, a complex
exercise is either correct or incorrect. For students
this is often frustrating. A tiny error and their almost
working solution is not accepted. When using such
a system at our university we observed that it works
well for the top students in a class. Weaker students,
however, quickly loose their motivation and do worse
than with conventional assignment sheets.
3 ASSESSMENT STRATEGIES
FOR ONLINE COURSES
For online courses, we need elements from both tutor-
ing systems and automated grading tools. That is, we
want to support and guide students when new con-
cepts are introduced. On the other hand, we want
them to solve more complex exercises and evaluate
these automatically.
In this paper, we propose a first step in this di-
rection. Our system makes it possible to define ex-
ercises of varying complexity. We allow teachers to
define exercises with step-by-step instructions simi-
lar to quizzes. But we also allow them to define more
comprehensive exercises where students may practice
algorithmic problem solving skills.
In order to accomplish this goal, we analyze the
source code using a unit testing framework. With unit
tests, we can test both program parts (e.g. a single
method or function) or the program as a whole. Unit
tests are on the one hand a familiar tool to any pro-
grammer and thus to any computer science teacher.
On the other hand, they allow teachers lots of flexibil-
ity when designing their exercises.
The idea is to define many unit tests for a single
exercise and test program parts instead of the over-
all functionality. As a consequence, we can return
a more fine-grained feedback. Not only is this feed-
back helpful for students when solving the exercises,
unit test also allow to attribute points to the individual
parts of an exercise. We are convinced that this leads
to a more transparent and fair grading result.
In the following sections we will show two small
sample exercises as they have been implemented in
our system.
CSEDU2014-6thInternationalConferenceonComputerSupportedEducation
422