MEASURING THE QUALITY OF DISTANCE EDUCATION
BETWEEN TWO CONTINENTS
Olayele Adelakun and Delvin Grant
DePaul University, Chicago, IL, U.S.A.
Keywords: Total Quality Management, Distance learning.
Abstract: The purpose of the paper is to measure the quality of distance learning education between CDM Chicago
and CDM Amman Jordon. To accomplish this task we used a Total Quality Management (TQM) Model
developed by Mergen et al. (2000) and Widrick et al. (2002) which consists of three dimensions (Quality of
Design, Quality of Conformance, and Quality of Performance). A case study approach was used to collect
data in Amman, Jordan and Chicago, USA. One key finding is that distance learning systems (IT and
business processes) in the US cannot be applied in Jordan without significant changes to the administrative
process. Another is that, while technology is global in nature, local resources and cultural issues make
distance learning difficult to adopt across continents.
1 INTRODUCTION
Quality has many definitions: Leavitt (1972) has
defined it as conformance to specifications, Crosby
(1979) defined it as conformance to requirements,
Juran (1988) defined it as fitness for us , and
Parasuraman, Zeitaml and Berry, (1985) have
defined it as meeting and/or exceeding customers'
expectations. Each definition has strengths and
weaknesses with respect to measurement,
generalization, managerial usefulness and consumer
relevance, therefore each is appropriate under
specific circumstances. Several authors have pointed
out that what is needed is a framework that
combines many of these dimensions (Wang and
Strong, 1996; Grant et al., 1999; Adelakun, 1999
and Widrick, et al. 2002). In this paper the quality
framework by Widrick et al. (2002) and Grant et al.
(1999) is adopted. The application of quality
measurement (QM) principles to solve industry-
related problems has been institutionalized by
institutions such as Xerox and Rochester Institute of
Technology to gain competitive advantage (Widrick
et al., 2002). Quality management plays an
important role in the daily operations and
management of these companies.
The College of Computing and Digital Media
(CDM) has one of the largest graduate programs in
computer science, telecommunications and
information systems in the US. It is part of DePaul
University located in Chicago. Over the past few
years CDM has expanded internationally to offer
graduate degrees in Poland and Jordan. However,
this study focuses exclusively on the CDM Jordan
program. We believe that the formal application of a
TQM framework will bring discipline to distance
learning education between the two programs and
will serve as means for continuous improvement.
Therefore, the goal of this paper is to propose a set
of measures and tools to evaluate each of the three
dimensions of quality (Quality of Design, Quality of
Conformance, and Quality of Performance) with
respect to the delivery of distance learning education
between CDM Chicago and CDM Amman Jordan.
The remainder of the paper is organized as follows.
Section two discusses the quality framework
adopted from Widrick et al. (2002). Section three
explains the set of proposed measures, tools and
techniques for measuring each of the three
dimensions of quality. The next section presents a
case study on distance education in Jordan. The
paper ends with discussion and conclusion.
2 QUALITY FRAMEWORK
Juran (1951) takes a customer centric approach in
defining quality as “fitness for use”. Juran (1993)
323
Adelakun O. and Grant D. (2009).
MEASURING THE QUALITY OF DISTANCE EDUCATION BETWEEN TWO CONTINENTS.
In Proceedings of the First International Conference on Computer Supported Education, pages 323-328
DOI: 10.5220/0001961803230328
Copyright
c
 SciTePress

later introduced planning, control and improvement
(Juran’s Trilogy) as part of the quality improvement
process. Grant et al. (1999) and Widrick et al. (2002)
build on Juran’s quality trilogy as follows: plan
(Quality of Design), control (Quality of
Conformance) and improve (Quality of
Performance).
Figure 1: Relationship between components of the model
(Grant et al., 1999).
Figure 1 suggests that the quality management
framework (QMF) cycle begins with the design
phase followed by a conformance stage and
concludes with the performance stage. Theoretically
the TQM process has no start or end point because
of its continuous improvement notion. Like all
processes there is a starting point which is assumed
to be the design stage. Issues of conformance or
performance may influence changes to the design
stage.
2.1 Quality of Design
Grant et al. (1999) noted that design quality deals
with determining the characteristics of a product or
service at a given cost. Widrick et al. (2002) pointed
out that the Quality of Design has to do with how
well the design captures the intended requirements.
The following three factors were used as a
determinate for measuring design quality: (1) design
insight and depth of requirements understanding; (2)
the quality of the process used to translate the
insights and requirements into valuable products or
services for the end consumer; (3) the continuous
improvement of the design process.
2.2 Quality of Conformance
Quality of Conformance describes how well firms
and their suppliers conform to the design
requirements. Conformance to requirements is one
of the earlier measures of quality (Crosby, 1979).
Quality of Conformance is relatively easier to
measure compared to Quality of Design or Quality
of Performance. While conformance quality is
relatively easy to measure, in higher education
distance learning it is difficult to measure partly due
to the subjectivity of the product or service and the
relatively long lead-time between delivery of service
and the realization of the rewards or benefits.
2.3 Quality of Performance
Performance quality is often expressed as fitness for
use, user satisfaction or meeting and/or exceeding
customer’s expectation (Juran, 1988; Grönroos,
1983). Widrick et al. (2002) explain that Quality of
Performance deals with how well a service or
product performs in the eyes of the consumer who
could be internal or external. It is important to point
out that each customer has a different assessment of
quality. For example, executives tend to look at cost
and return on investment as a measure of IT
investment, while technical personnel measures
things like IT downtime and system functionality.
Performance
quality is one area of assessment
that is often not measured in distance learning.
3 APPLICATION OF THE
FRAMEWORK:
MEASUREMENT
PARAMETERS AND TOOLS
Ensby, et al. (1997) proposed that the Baldrige
quality criteria are appropriate for assessing
functions of higher education institutions. Baldrige
criteria emphasize identifying and satisfying
customer needs. Ensby also suggested that students
are the product of higher education institutions and
the employers are the customer. Viewing students as
products is different from other views taken by
researchers such as Bailey et al. (1996) and Kanji et
al.
(1998) who believe that university program artifacts
such as curriculum, courses, projects, and so on are
the products and students are the customers. Grant et
al. (1999) suggest that university faculties and
administrators should listen to
CSEDU 2009 - International Conference on Computer Supported Education
324

Table 1: Parameters and Tools.
Framework Measurable parameters Measurable tools
Quality of Design Online education needs and Market potential
Online resources and technical skill to deliver
What program will be included in the distance
education
Admission requirements
Continues training for technical staff
Knowledge and skill requirements of the distance
education locations
Cultural and environment understanding of global
delivery.
Survey of voice of customer
Interview and meeting with stakeholders at
offsite location
Identify offsite stakeholders at various levels.
Focus group on program requirements and
technical requirements
Identify and plan for administrative resources
at the local site
Survey and Plan of capable instructors for the
distance program.
Distance learning (DL) program committee
Quality of Conformance List of degree programs in the online education
program
Available and fully functional technical resources
Level of courses coverage in the onsite program
and the distance learning program.
Are courses offered in proper sequence in both
program
Involvement of qualified instructors in the online
program
Confirmation of DL degree program
Confirmation of tech resources
Confirmation of level of coverage
Confirmation of sequence of courses offered
Confirmation of qualified instructors
Quality of Performance Graduation level from the online education
program
Students satisfaction of the online education
Employers satisfaction and continuous
recruitments from the online program
Alumni positions and compensation over their
career cycle.
Returning alumni for continues education and
career development.
Graduation analysis
Satisfaction survey and satisfaction trend
analysis
Industry feedback
Alumni feedback
New students from the employers
the concerns of employers who are the true
customers and take them into consideration when
designing courses for students who are the product.
This view served as a guide to identify several
measurement parameters and tools that could be
applied to distance learning (see table 1).
4 CASE STUDY
One of the co-authors spent six weeks in Jordan
interviewing several stakeholders involved in the
distance education partnership between CDM
Chicago, and CDM Amman, Jordan. The co-author
also taught classes in Jordan which allowed him to
interact with students and ask them about their views
of distance learning. Until the spring 2008, CDM
was formally known as School of Computer
Science, Telecommunication and Information
Systems (CTI). Since early 2000, CDM is the largest
graduate school in the US. This is partly due to the
breath and quality of the degree programs that are
offered. Currently CDM offers twelve undergraduate
degrees and fifteen graduate degrees in fields
ranging from traditional Computer Science to
Multidisciplinary Studies. Examples of
nontraditional degree programs include Human-
Computer Interaction, Digital Cinema, Project
Management, and Interactive Media. CDM is one of
the few computer science schools in US that offer a
comprehensive degree program in Digital Media,
Gaming, and Animation.
MEASURING THE QUALITY OF DISTANCE EDUCATION BETWEEN TWO CONTINENTS
325

4.1 Design Quality
The dean of CDM in early 2000 was very aggressive
in developing new programs and expanding the
highly successful CDM brand internationally. One
of the countries of interest was Amman Jordan. To
start distance education in Jordan two visiting
professors were hired from Jordan.
The design quality focused on planning and defining
the key requirements of the Jordan program. One of
the visiting faculty members from Jordan was
instrumental in the design of the program which is
physically hosted at the partner University in Jordan.
The design team is made of personnel from CDM
Chicago, one of the visiting professors, the dean, and
members from the Jordanian Ministry of Education.
The design team discussed the long term market
needs of the program, the immediate programs
that could be delivered online, and those that could
be added later, faculty members that would be
involved with the program, and the technology
requirements. The online program proposal was
submitted to the university committee and it was
approved for implementation. Based on feedback
from the Jordanian Ministry of Education changes
were made to the final online delivery proposal.
One key ingredient to the design was the home
grown online software (DL-Web) that enables
students in Jordan to login to DL-Web to play videos
of the classes in Chicago. Lectures at the Chicago
campus are available online to Jordanian students a
few hours after classes end in Chicago. Jordanian
students actually take the same classes as their
Chicago counterparts via the DL-Web software. A
local instructor in Jordan provides local tutorial
support and helps with explanation of key course
concepts.
The focus of the design phase is how the Chicago
courses would be delivered in Jordan. Since Chicago
has been providing distance education to students
outside Chicago via DL-Web, it was expected that
time and distance would not be a barrier. Faculty
members teaching online courses in Jordan could
interact with students through the DL-web
discussion forum and email.
The last set of design issues was about the local
activities and processes involved in setting up the
Jordan campus, like buying tables, chairs, and
setting up the computer network. The objective was
to make the product available internationally where
Jordan would be one of several international venues.
Base on interviews with local administrators we
understood that input from potential employers were
not significantly solicited at the design phase.
4.2 Conformance Quality
All of the design requirements were met by the end
of the first year. However, issues that were not
defined in the requirements came up later. For
example, cultural issues and management were areas
that needed to be addressed. Other unforeseen
technical issues immerged so IT personnel and CDM
administrators from Chicago visited Jordan campus
frequently to ensure that the program would function
technically and administratively. We observed that
all the computers in the labs were functional and the
course technology equipment worked. We were able
to login to DL-Web and view course lectures,
assignments, grades and so on. One change that was
made from the original design was that students can
pick up class lecture DVDs in addition to DL-Web
because many Jordanian students don’t have access
to high speed internet at home. Exams are proctored
locally and mailed to Chicago for grading. Take-
home assignments can be submitted online and the
instructors’ comments on student assignments are
submitted online. In addition, faculty members
occasionally travel to Jordan to teach. Conformance
was continuously monitored by soliciting feedback
from students on the program, especially on course
quality, teaching, and course usefulness.
Consequently, conformance quality measurements
have been focused on program elements including
content delivery to ensure that it does not vary
significantly from the Chicago program.
4.3 Performance Quality
Grant el at. (1999) and Ensby et al. (1997) agree that
performance quality is one area that is seldom
measured and unfortunately, the same is true in this
case study. However, we are hoping to correct this
limitation in the near future by measuring the
performance of the program beyond student
feedback. Most universities seldom considered the
employer as the final customer hence they seldom
measure the performance of their product (the
students) in conjunction with the end customer
(employer). The only ad-hoc assessment of
performance quality was comments from the
students. One student commented that having taking
the capstone course he was able to understand what
he needs to do to become the CIO of the company he
works for. Several students completing the program
are considering changing jobs and careers after
CSEDU 2009 - International Conference on Computer Supported Education
326

Table 2: Proposed Quality Assessment.
acquiring new knowledge through the program. One
student was considering starting an offshore
outsourcing company. Efforts are on the way for the
program to collaborate with industry partners. For
example managers from IT companies including
IBM Jordan were invited as guest speakers.
5 DISCUSSION/CONCLUSIONS
In this paper we identified a set of measurement
parameters and tools to evaluate the three
dimensions of quality with regards the distance
learning between CDM Chicago and CDM Jordan.
The proposed framework should prove useful for
higher education institutions considering distance
learning education implementation and assessment.
If we follow the TQM process proposed by Grant et
al. (1999) and Ensby et al. (1997) we would
conclude that the program is not the product but the
students are. This view may force CDM Chicago to
reevaluate the final product, is it curriculums and
programs, students, or employers? In Chicago, CDM
has a board of directors made up of industry IT
executives. A similar approach should be introduced
in Jordanian. Jordanian Board members could
provide external review and valuable feedback of
industry needs to design programs to better meet the
need of customer (employers). They should establish
a Jordanian alumni association to provide advice and
aid the collection of Quality of Performance
assessment. While the US-Jordan program has been
very successful in its program delivery, several
lessons were learned.
First, there is a difference between distance learning
within the US (Chicago-New York) and distance
learning across national boundaries or continents.
Some of the assumptions that work in the US
certainly do not work globally. For example, most
students at DePaul have a PC with high speed
internet connection at home. This is not true for
Jordanian students.
Second, IT system that works well in one cultural
environment may not work in another (Ruohonen
and Adelakun, 1997). This is true for all aspects of
IT and other products. If CDM defines their product
to be the programs, projects, curriculums and the
DL-Web application it may present some challenges
because research has shown that education has to be
tailored to the needs of the society. While theories
are global in nature the application is local,
therefore, performance measurements have to
address how the theories are applied locally. The
current dean of CDM is taking steps to include
industry partners in the improved design of the
Jordanian program. He has approved to fund a career
day event that would involve several Jordanian IT
companies. Based on the findings of this research we
propose a new view of quality for CDM (Table 2.).
Third, traditionally there seems to be a gap between
industry and academia in terms of what subject
matter and needed skills are important for student
success. Many universities are involving industry
experts in curriculum development. A good example
of this partnership is the development of the MS
degree in Manufacturing Management and
Leadership at Rochester Institute.of Technology in
USA (Widrick et al., 2002). Such partnership fosters
stronger ties between academia and industry. CDM
could use such a relationship as another source for
student recruitment into the program. This is
extremely important due to the cultural and IT
differences between the two countries. Cultural
issues did not pose significant challenges because
the theories discussed in the degree program were
the same in Chicago and Jordan. The key differences
were in the application of theory to practice, the
resources available for implementation, and
technology
Method Quality of Design Quality of Conformance Quality of Performance
Action
Research
CDM should focus the design on
customer expectations (i.e. IT
needs within the Jordanian
industrial environment). Such focus
and understanding will be used to
determine which CDM program
will be adjusted and offered in
Jordan. It will also determine which
skill set is required for local
instructors.
Conformance qualities would be
assessed through continuous
evaluation processes on classes,
the instructors and the online
technological fit.
Formal and informal feedback from
the industry will be obtained as part
of the quality process. Some of the
industrial participants will include
CDM alumni and board of directors.
Comments from this group will feed
into the design and conformance
quality assessment.
MEASURING THE QUALITY OF DISTANCE EDUCATION BETWEEN TWO CONTINENTS
327

Fourth, administration of distance learning across
continent is more challenging than distance learning
within the US. Time difference, language, religion,
and social norms present administrative challenges
to distance education in Jordan. In case of CDM, a
permanent on site CDM director was available at the
offsite location. The director in Jordanian has a
strong US academic background and was an
instructor in CDM for several years before assuming
the position in Jordan. Such local and global
knowledge was instrumental to the success of the
joint program. If it were not for his local knowledge,
the cultural issues would have derailed the
successful implementation of the program.
Lastly, performance measurement is one area that is
often problematic and seldom measured and this was
confirmed in our study. In many universities
performance quality is assessed by measuring the
number of students in a program and revenue from
the program. These are typical ways of evaluating
the performance quality but they provide a limited
assessment of performance quality. Future
performance measurement should include customer
evaluation of the product performance (CDM
Jordanian graduate students). We believe this is the
only way to maintain sustainable competitive
advantage in the long run. Using TQM proposed by
Widrick et al. (2002) performance issues could be
used as a trigger for improving the design and
conformance of the program. Future research may
be used to survey Jordanian alumni and industry
participants on their assessment of Jordanian
graduates (i.e. the end products). Another study may
be used to assess CDM distance learning students in
US.
REFERENCES
Adelakun, O., (1999). Information Quality Process – A
Primary Success Requirement in Management
Information Systems Projects: Results from a Case
Study, Proceedings of the Fifth Americas Conference
on Information Systems (AMCIS) ,Milwaukee,
Wisconsin on August 13-15, 1999.
Bailey, Duncan, and Jerome V. Bennett (Nov. 1996). “The
Realistic Model of Higher Education” Quality
Progress, pp.77-79
Baskerville, R. (1999). Investigating Information Systems
with Action Research, Communications of the
Association of Information Systems, Vol. 2, Article,
19. Also available online @
http://www.cis.gsu.edu/~rbaskerv/CAIS_2_19/CAIS_
2_19.html.
Carnegie, A. (1920). Autobiography of Andrew Carnegie,
New York: Houghton Mifflin.
Crosby (1979). Quality is free: The Art of Making Quality
Certain, Published by McGraw Hill Book Company,
New York, USA.
Ensby, M., and F. Mahmoodi (April 1997). “Using the
Baldrige Award Criteria in College Classrooms,”
Quality Progress, pp.85-91.
Grant, D., Mergen, E., and Widrick, S. (1999). An analysis
of quality measurement approaches in higher
education. International Association of Computer
Information Systems proceedings, pp. 70-76.
Juran, J.M. (1951). Quality Control handbook, McGraw-
Hill, New Jersey
Juran, J.M. (1988). Juran on Planning for Quality. New
York: Free Press.
Juran, J.M. and Gryna, F.M., (1993). Quality Planning and
Analysis, third edition, McGraw-Hill, pp. 7-10.
Kanji, Gopal K. and Abdul Malek A. Tambi (1998).
"Total Quality Management and Higher Eduation in
Malaysia" Total Quality Management Vol. 9,
Numbers 4 & 5, pg. 130-132
Leavitt, T. (1972). Production-Line Approach to Service,
Harvard Business Review, Vol. 50, No. 5, pp.41-52.
Mergen, E., Grant, D, and Widrick, S., (2000). Quality
Management applied to higher education, Total
Quality Management, 11, pp. 345-352.
Parasuraman, A., Zeithaml, V.A., and Berry, L.L. (1985)
A conceptual Model of Service Quality and Its
Implication for Future Research. Journal of
Marketing, Vol. 49, No.4, pp.41-50.
Rapoport, R.N. (1970). "Three Dilemmas in Action
Research," Human Relations, Vol 23, No. 6, pp. 499
513.
Robert M Pirsig, (1974). Zen and the Art of Motorcycle
Maintenance, Bodley Head, London.
Ruohonen, M. and Adelakun, O. (1997). The Need and
Challenges of Information Systems Education from a
Nigerian Perspective. In Capacity Building for
Information Technologies in Education In Developing
Countries, Marshall G. and Ruohonen M. (Eds.). pp.
175-184, North-Holland. Netherlands
Wang, R. Y., and Strong, D.M., (1996). Beyond
Accuracy: What Data Quality Means to consumers,
Journal of MIS, Vol. 12, No.4, pp.5-34.
Widrick, S., Mergan, E., and Grant, D. (2002). Measuring
the dimension of quality in higher education, TQM,
Vol. 13, No. 1, pp. 123-131.
CSEDU 2009 - International Conference on Computer Supported Education
328