![](bg4.png)
node {C, −B, A, −C}: {A,−B,C, −C} with descrip-
tion ∆ = ((0,1), (1,0), (1,1)). We will compute this
transformation for both parent nodes. Then we say
that a node is smaller if the description ∆ is smaller
with respect to the Ω ordering applied lexicographi-
cally per components. Continuing with our example
above, let the second node be {B,C, A, −A} with ∆ =
((0,1), (0,1), (1,1)). Then this second node is smaller
than the first node {A, −B,C, −C}, since the first com-
ponents are equal and (1,0) is greater than (0,1) in case
of second components.
4.2.3 Generalization of Subgraphs
Now we can describe how the generalized graphs are
built. After the reordering introduced in the previous
paragraph, we assign variables Z,Y,X,W,V,U,. . . to
propositional letters. Initially, we merge literals
from all nodes into one list and order it using the
Ω ordering. After that, we assign variable Z to the
letter with the smallest value, variable Y to the letter
with the second smallest value, etc. If two values
are equal, we compare the corresponding letters only
within the first parent, alternatively within the second
parent or child, e.g., for the student’s (incorrect)
resolution step {C, −B, A, −C}, {B,C, A, −A} →
{A,C}, we order the parents getting the re-
sult {B,C, A, −A}, {C, −B, A, −C} → {A,C}.
Next we merge all literals into one list
{B,C, A, −A,C, −B, A, −C, A,C}. After reordering,
we get {B,-B,C,C,C,-C,A,A,A,-A} with ∆ = ((1,1),
(1,3), (1,3)). This leads to the following renaming of
letters: B → Z, C → Y , and A → X . Final generalized
subgraph is {Z,Y, X, −X}, {Y, −Z, X , −Y } → {X,Y }.
In the case that one node contains more propositional
letters and the nodes are equal (with respect to the
ordering) on the intersection of propositional letters,
the longer node is defined as greater. At the end, the
variables in each node are lexicographically ordered
to prevent from duplicities such as {Z, −Y } and
{−Y, Z}.
4.3 Classification of Correct and
Incorrect Solution
For testing of algorithm performance we employed
10-fold cross validation. At the beginning, all student
solutions have been divided into 10 groups randomly.
Then for each run, all frequent three-node subgraphs
in the learning set have been generated and all gener-
alization of those subgraphs have been computed and
used as attributes both for learning set and for the test
fold. The results below are averages for those 10 runs.
We used four algorithms from Weka package, J48
decision tree learner, SMO Support Vector Machines,
IB1 lazy learner and Naive Bayes classifier. We ob-
served that the best results have been obtained for
minimum support below 5% and that there were no
significant differences between those low values of
minimum support.
The best results have been reached for general-
ized resolutions subgraphs that have been generated
from all frequent patterns, i.e. with minimum sup-
port equal to 0% found by Sleuth (Zaki, 2005). The
highest accuracy 97.2% was obtained with J48 and
SMO. However, J48 outperformed SMO in precision
for the class of incorrect solutions—98.8% with re-
call 85.7%. For the class of correct solutions and J48,
precision reached 97.1% and recall 99.7%. The aver-
age number of attributes (generalized subgraphs) was
83. The resulting tree is in Fig. 3. The worst perfor-
mance displayed Naive Bayes, especially in recall on
incorrect solutions that was below 78%. Summary of
results can be found in Table 1.
If we look at the most frequent patterns which
were found, we will see the coincidence with patterns
in the decision tree as the most frequent pattern is
{Z}, {−Z} → {#} with support 89%. This pattern is
also the most frequent for the class of correct proofs
with support 99.7%, next is {Y, Z}, {−Y, Z} → {Z}
with support 70%. Frequent patterns for incorrect
proofs are not necesarilly interesting in all cases. For
example, patterns with high support in both incorrect
and correct proofs are mostly unimportant, but if we
look only at patterns specific for the class of incorrect
proofs, we can find common mistakes that students
made. One such pattern is {Y, Z}, {−Y, −Z} → {#}
with support 32% for the incorrect proofs. Other sim-
ilar pattern is {−X ,Y, Z}, {−Y, X } → {Z} with sup-
port 28%.
4.4 Complexity
Complexity of pattern generalization depends on the
number of patterns and the number of literals within
each pattern. Let r be the maximum number of literals
within a 3-node pattern. In the first step, ordering of
parents must be done which takes O
r
for counting
the number of negative and positive literals, O
r log r
for sorting and O
r
for comparison of two sorted
lists. Letter substitution in the second step consists
of counting literals on merged list in O
r
, sorting the
counts in O
r log r
and renaming of letters in O
r
.
Lexicographical reordering is performed in last step
and takes O
r log r
. Thus, the time complexity for
whole generalization process on m patterns with du-
plicity removal is O
m
2
+ m(4r + 3r log r)
.
CSEDU2014-6thInternationalConferenceonComputerSupportedEducation
566