the CRAS properties. The t-test was intended to com-
pare the performance of the part-time students (the ex-
perimental group) with that of the 47 full-time DBA
students (the control group) who had used the stan-
dard module text (Bernstein et al., 1987), henceforth
denoted by B, but not ITSTP. The full-time students
had taken the same test in the semester before the ex-
perimental group. Secondly, a 5-point Likert scale
was used to produce data on the perceptions the part-
time students had of ITSTP and B, as methods for fa-
cilitating understanding of the CRAS properties, and
their attractiveness as learning instruments. A t-test
was again used to analyse the data produced and was
based on a comparison of the matched pairs of scores
produced by each respondent for ITSTP and B.
Mean test scores for the part-time and full-time stu-
dents for the phase test were calculated for both the
CRAS and non-CRAS related questions to compare
the performance of the two sets of students. Because
we felt that ITSTP might have had an effect in encour-
aging learning gains, we chose to test the alternative
(directional) hypothesis that: the part-time students
performed better on the test of CRAS property under-
standing than the full-time students. The correspond-
ing null hypothesis was that there was no difference
in the phase test scores produced by the two sets of
students.
The analysis of our data showed that the mean test
scores (out of 12) for the full-time and part-time stu-
dents on the questions on CRAS properties were 6.52
and 8.38 respectively (the corresponding standard de-
viations were 9.65 and 6.97, respectively). The com-
puted t-statistic was 0.81 for 27 degrees of freedom.
Hence, the directional hypothesis had to be rejected
in favour of the null hypothesis.
A comparison of the mean scores (out of 28)
achieved by the students on that part of the phase
test that did not directly relate to the CRAS properties
revealed that the average mark for full-time students
was 16.96 whereas the average mark for part-time stu-
dents was 17.58. As such, whereas the average score
for the full-time students on the phase test questions
relating to the CRAS properties was 29% lower than
the part-time students, the average mark for the full-
timers on questions not related to the CRAS proper-
ties was only 4% lower. Although these figures do not
prove anything, they offer some suggestive evidence
that ITSTP might have helped students to understand-
ing the CRAS properties.
The combination of the statistically non-significant
analysis of the phase test scores and the fact that a
number of potentially confounding variables applied
in our study meant that we were not able to draw any
firm conclusions on whether ITSTP had been of value
in terms of helping students understand the CRAS
properties.
The Likert scale included a total of 24 statements
(with an equal number of positive and negative state-
ments). These items were divided into three cate-
gories. Eight of the statements were intended to mea-
sure the extent to which ITSTP and B were perceived
as being of value in facilitating student understanding
of the CRAS properties, a further eight items were
intended to help to decide the extent to which ITSTP
and B were motivating to use, and the remaining eight
statements were used to collect the students’ opinions
on the value of comparable features of ITSTP and B
(i.e., their explanations, exercises and examples). Stu-
dents were asked to indicate their strength of agree-
ment/disagreement with each statement in the Likert
scale. The five options were: strongly agree, agree,
unsure, disagree and strongly disagree.
26 responses to the Likert scale were returned. To
produce the measures of student attitudes, a three-
stage approach was adopted. The initial step involved
“signing” the 24 items included in the Likert scale as
being a positive or negative statement about ITSTP or
B. Next, the returns were analysed using the follow-
ing system: for each positive statement a response of
“strongly agree” was given a score of 5, an “agree”
response was given a score of 4, a score of 3 corre-
sponded to an “undecided” response, “disagree” was
scored as a 2, and “strongly disagree” was scored
as a 1. Conversely, for each negative statement, a
“strongly agree” response was given a score of -5,
“agree”was scored as -4, “undecided” was recorded
as a -3, “disagree” was given a score of -2, and -1 cor-
responded to a “strongly agree” response. By sum-
ming the scores for each return, a figure correspond-
ing to the respondent’s attitude towards ITSTP and B
was computed. In the final step, the B score for each
respondent was subtracted from the score for ITSTP.
This calculation gave a measure of a respondent’s atti-
tude to ITSTP that is relative to their attitude towards
B.
3
To analyse the information produced from the Lik-
ert scale, t-statistics were computed to compare the
mean scores for the perceptions students had of IT-
STP and B, overall and for each of the three categories
of items included in the Likert scale.
In the overall measure of the two methods, the av-
erage difference in the ratings of ITSTP and B was
16.79 in favour of ITSTP, and no student reported that
B was “better” than ITSTP. The t-statistic for the com-
parison of average differences was 2.25. This is sta-
tistically significant at the 2% level. Not surprisingly,
given the overall results, ITSTP was also perceived to
be “better” than B in all three of the sub-categories of
Likert scale items.
In terms of facilitating understanding of the CRAS
3
A positive score indicates a more favourable attitude
towards ITSTP than B; a negative score represents a more
favourable attitude towards B than ITSTP.
ICEIS 2004 - ARTIFICIAL INTELLIGENCE AND DECISION SUPPORT SYSTEMS
202