![](bg2.png)
course delivery particularly useful and
advantageous. The web-based PBL involves the
creation of instructional materials that facilitate
problem presentation; the required self-
investigations and analysis can be conducted
through online resources and the social interaction
for peer-peer collaboration and student-teacher
facilitation can likewise be easily performed through
asynchronous forum boards or synchronous chats.
The final presentation of the answers to the problem
can also be easily conducted through the array of
educational media technology tools made available
by CMC.
There have been several studies that attempt to
evaluate the effectiveness of web-based PBL
(Norman and Schmidt, 1992) Reported that students
perceive the web-based PBL to be motivating,
providing access to greater richness of resources and
developing collaborative networks; the skills that
they subsequently acquire are appropriate for their
future professional activities. (Orill, 2002) Revealed
that with a higher degree of student control related to
aspects such as content, the instructional path, pace
and feedback, the PBL approach contributes
significantly to improved cognitive gains. (Ronteltap
and Eurelings, 2002) Highlighted that the
combination of synchronous and asynchronous tools
in the web-based PBL leads to deeper levels of
information processing when students compose
documents that represent their personal knowledge
based on their research from online resources.
The importance of the web-based PBL especially for
course delivery in distance and open learning, a
study that attempts to evaluate the effectiveness of
this approach relative to the widely available online
Content-Based Learning (CBL) both in terms of
students’ performances and perceptions (Gijselaers
and Segers, 2002).
According to (Chickering and Gamson, 1991) and
Chickering and Ehrmann, 1996), positive online-
learning environments incorporate seven principles
of good teaching practice. These should include:
• encouraging students-faculty contact,
• encouraging cooperation among students,
• encouraging active learning,
• giving prompt feedback,
• emphasizing time on task,
• communicating high expectations, and
• respecting diverse talents and ways of
learning.
One important outcome of any educational effort is
to prepare the candidate to face the challenges in the
working environment. How can that be measured?
Donald Kirkpatrick’s model can be used to address
the knowledge, skill & abilities needed to
successfully apply the online learning material in the
“real world” (Kirkpatrick, 1996). Kirkpatrick’s four
evaluation measures are reaction, learning, behavior,
and results.
For the evaluation of the relative effectiveness web-
based PBL and traditional PBL we will find the
answers of the following questions:
1. Effectiveness of the web-based PBL
environment compared with traditional
PBL in terms of students academic
achievements
Table 1: Kirkpatrick’s four levels (Alliger, et al. 1998, Tannebaum 1998).
Level Definition Questions Addressed Guidelines for Measuring Each Level
Level 1:
Reaction
Assesses participants’ initial reactions to a course.
Offers insights into participants’ satisfaction, or the
effectiveness (value) of the training as perceived by the
trainee. Usually assessed through a survey aka a
"smiley sheet."
Does not measure learning.
Were the participants pleased?
What do they plan to do with what they learned?
Determine what you want to find out.
Design a form that allows questionnaire results to be easily tabulated.
Encourage honest written comments and suggestions. Attain an immediate
response rate of 100 percent.
Develop standards.
Measure reactions against the standards and take appropriate action.
Communicate participants’ reactions.
Use focus groups to acquire qualitative feedback (i.e. more specific comments)
Level 2:
Learning
Assesses the amount of information (principles, facts
and techniques) understood and absorbed by trainees.
May use a criterion-referenced test
What skills, knowledge, or attitudes have
changed? By how much?
Use a control group, if feasible.
Evaluate knowledge, skills, or attitudes both before and after the training.
Attain a response rate of 100 percent.
Use the results of the evaluation to take appropriate action.
Level 3:
Behavior,
or Transfer
Assesses the amount of material used on-the-job after
taking the course, e.g. a week to 6 months (or longer)
after taking the course. Assesses on-the-job behavior
based on the objectives of the course and assessed
through tests, observations, surveys and interviews.
Did the participants change their behavior based
on what was learned in the program?
Use a control group, if feasible.
Allow enough time for a change in behavior to take place.
Survey or interview one or more of the following groups: trainees, their bosses,
their subordinates, and others who often observe trainees' behavior on the job.
Choose a statistically significant sample, or 100 employees.
Repeat the evaluation.
Consider the cost of evaluation versus the potential benefits.
Level 4:
Business
results
Measures results, e.g. reduced costs, higher quality,
increased production, and lower rates of employee
turnover. Measure 6 mos. to 2 yrs. after completing the
course.
Did the change in on-the-job behavior positively
affect the organization?
Use a control group, if feasible.
Allow time for results to be achieved. Amount of time depends on course
context.
Measure both before and after training. Repeat the measurement.
Consider the cost of evaluation versus the potential benefits.
EFFECTIVENESS OF WEB BASED PBL USING COURSE MANAGEMENT TECHNOLOGIES: A CASE STUDY
171