![](bg4.png)
ated. With this technique, the agent evaluates a small
set of trees when making a decision. The trees in the
set are dynamically created for better space efficiency.
In this section, we discuss the grouping of trees, and
in the next section, the dynamic tree creation.
The discussion is in the context of an experimen-
tal ITS, which we developed for testing our tech-
niques. The ITS teaches concepts in software basics.
A POMDP helps the ITS choose optimal teaching ac-
tions. We cast the ITS on to the POMDP, by using
POMDP states to represent student knowledge states,
and actions to represent system tutoring actions. We
treat student actions (asking questions, accepting an-
swers, etc ) as observations.
4.2 State Space Partitioning
To have small tree sets, we partition the state space
into subspace, and then group trees in each subspace
into tree sets. Before the tree grouping technique, we
discuss our method for state space partitioning.
We define the states in terms of concepts in the
instructional subject. In software basics, concepts in-
clude program, instruction, algorithm, and many oth-
ers. We associate each state with a state formula,
which is of the form:
(C
1
C
2
C
3
...C
N
), (8)
where C
i
is the variable for the ith concept C
i
, taking
a value
√
C
i
or ¬C
i
(1 ≤i ≤N), and N is the number
of concepts in the subject. We use
√
C
i
to represent
that the student understands C
i
, and ¬C
i
to represent
that the student does not. A formula is a representa-
tion of a student knowledge state. For example, for-
mula (
√
C
1
√
C
2
¬C
3
...) is a representation of the state
in which the student understands C
1
and C
2
, but not
C
3
, ... States thus defined have Markov property.
When there are N concepts in an instructional sub-
ject, the number of state formulae is 2
N
. This implies
that the number of possible states is 2
N
. As can be
seen in Eqn (2), the cost for evaluating a value func-
tion is proportional to the size of state space. To re-
duce the cost, we partition the state space into smaller
subspaces. The partitioning technique is based on
prerequisite relationships between concepts.
Prerequisite relationships are pedagogical orders
of concepts. A concept may have zero or more pre-
requisites, and a concept may serve as a prerequisite
of zero or more concepts. For example, in mathemat-
ics, derivative has prerequisites function, limit and so
on, and function is a prerequisite of derivative, inte-
gral, and so on. To understand a concept well, a stu-
dent should understand all its prerequisites first. For
a set of concepts, the prerequisite relationships can be
represented by a directed acyclic graph (DAG). In this
paper, when a concept is a prerequisite of another, we
call the latter a successor of the former.
In the first step of our space partitioning tech-
nique, we subdivide concepts such that concepts hav-
ing prerequisite relationships are in the same group.
Some very “basic” concepts may be in two or more
groups. In the second step, for each group, we cre-
ate a state subspace by using concepts in the group to
define states, in the way just discussed. In the third
step, we eliminate invalid states. For details of the
partitioning technique, please see (Wang, 2015). Af-
ter space partitioning, we create policy trees for each
subspace. In a tree, the nodes and edges concern con-
cepts in the subspace only.
This partitioning technique is based on our obser-
vation that in a window in a tutoring process, stu-
dent questions likely concern concepts that have pre-
requisite relationships with each other. The observa-
tion suggests that we could localize the computing for
choosing an optimal teaching action within a smaller
state subspace defined by concepts having prerequi-
site relationships.
The total number of states in the subspaces are
much smaller than the number of states in the space
defined by using all the concepts. In addition, the
number and sizes of policy trees in subspaces are
much smaller because the sets of actions and obser-
vations are smaller.
4.3 Policy Tree Grouping
The cost for making a decision depends on the num-
ber of trees to evaluate, that is, the size of T in Eqn
(5). For lower costs, we group the trees in each sub-
space into small tree sets. When choosing an optimal
teaching action, the agent evaluates trees in a single
set. For discussing the grouping method, we first de-
fine optimal action and tutoring session.
In science and mathematics subjects, many con-
cepts have prerequisites. When the student asks about
a concept, the system should decide whether it would
start with teaching a prerequisite for the student to
make up some required knowledge, and, if so, which
one to teach. The optimal action is to teach the con-
cept that the student needs to make up in order to un-
derstand the originally asked concept, and that the stu-
dent can understand it without making up other con-
cepts.
A tutoring session is a sequence of interleaved
student and system actions, starting with a question
about a concept, possibly followed by answers and
questions concerning the concept and its prerequi-
sites, and ending with a student action accepting the
CSEDU 2018 - 10th International Conference on Computer Supported Education
140