LEARNER SATISFACTION WHEN APPLYING AN

INSTRUCTIONAL MODEL IN E-LEARNING

An Experimental Study

Fernando Alonso, Genoveva López

Department of Languages and Systems and Software Engineering, Facultad de Informática-UPM, Madrid, Spain

José María Font, Daniel Manrique

Department of Artificial Intelligence, Facultad de Informática-UPM, Madrid, Spain

Keywords: Instructional model, Distance learning, Web-based distance education, Learner satisfaction.

Abstract: This paper presents the results regarding the satisfaction of learners attending a course under the following

instructional conditions: traditional face-to-face classroom, distance learning without an instructional model,

having only virtualized the teaching contents used in face-to-face classrooms; and distance learning with an

instructional model. The courses on which the experiments were run are related to information and

communications technologies. Specifically, we present in this paper the results for a Java programming

course taught to information technology specialists under the above three instructional conditions. The

course was originally designed for classroom attendance. Later, with the aim of reducing student travel

expenses, all the course teaching content was virtualized (distance learning without an instructional model).

And the course was designed to simulate face-to-face classroom teaching via web and videoconferencing. In

view of the sharp drop in learner satisfaction with this second teaching mode, we adopted an instructional

model to systematize the distance course design and teaching process. The results suggest that learner

satisfaction in this study is slightly higher among students attending the distance course applying the

instructional model than for students taking the traditional face-to-face classroom course and greater in both

cases than among students enrolled in distance learning without an instructional model for the course.

1 INTRODUCTION

A good educational system should not focus

exclusively on transmitting knowledge from the

teacher to the learners (Govindasamy, 2002). It

should concentrate on the key aspect of training:

learning. Learning is the outcome of several separate

cognitive processes used to assimilate facts,

concepts, procedures, etc., and build new mental

representations of knowledge. These representations

can be applied in situations other than the settings

where they were learned and used to successfully

solve problems (Altenhofen and Schaper, 2002).

Learning is not just about acquiring new

knowledge, but also about consolidating,

restructuring or replacing what we already know. In

any case, it always leads to a change in the structure

of the brain, altering learners’ knowledge schemata

and/or cognitive structures. Learning is achieved by

accessing information, communicating

interpersonally with teachers or peers, and carrying

out cognitive operations (Pazos, Azpiazu et al,

2002).

Learning processes are activities carried out by

learners to achieve the educational objectives that

they aim for. It takes place through a process of

internalization where each learner accommodates

new knowledge in their existing cognitive structures

(Anderson, 1996). Conceptions about learning and

the roles learners should play in such processes have

evolved from learning being originally considered as

an acquisition and reproduction of informative data

transmitted by a teacher to now being viewed as a

construction or mental representation of meanings.

Over the last few years, there have been major

technological advances improving and easing

Internet-based distance education (Arriaga, El Alami

et al, 2003). This is what is known as e-learning. E-

141

Alonso F., López G., Font J. and Manrique D. (2010).

LEARNER SATISFACTION WHEN APPLYING AN INSTRUCTIONAL MODEL IN E-LEARNING - An Experimental Study.

In Proceedings of the 2nd International Conference on Computer Supported Education, pages 141-146

DOI: 10.5220/0002767601410146

 SciTePress

learning can be defined as the use of new

multimedia technologies and the Internet to improve

learning quality by easing access to resources and

services, as well as remote exchange and

collaboration (European Parliament, 2001) or the use

of network technologies to create, foster, deliver and

facilitate learning anytime and anywhere (Sampson

and Karagiannidis, 2002). However, all this

technology developed around distance learning is

useless without an instructional model to drive the

process of building and executing a distance course

as illustrated empirically in this paper, confirming

the results of earlier research (Tallent-Runnels,

2005; Alonso, López et al, 2005).

Our experience is based on teaching training

courses related to information and communications

technologies to information technology specialists

since the late 1990s. These courses were originally

designed to be taught as three- to five-hour

traditional face-to-face classroom sessions, each

course having a total workload of 20 to 50 hours.

The results were satisfactory not only in terms of

learning outcomes but also as regards learner

satisfaction (Alonso, López et al, 2008). The biggest

drawback for teaching these courses, though, was

the high travel and maintenance expenses they

generated, as the students came from all over Spain.

For this reason, it was decided, in view of the glut of

technological advances in the early 21

century, to

virtualize all the teaching material available for the

class attendance courses and give students Internet

access to these contents. This moved the courses into

the distance education domain. However, this option

failed to include an e-learning instructional model.

The learners attending the courses taught in this

mode were not happy with this move, and

satisfaction dropped sharply from 4.25 out of 5 on

average for the class attendance courses to 3.28 for

distance education with virtualized contents.

This drop in learner satisfaction was what was

behind the design of a web-based instructional

model for distance learning (Alonso, López et al,

2005; Alonso, Manrique et al, 2009). The

instructional model was to provide guidance not

only on how to prepare the educational contents to

be rendered for learners but also how to teach the

courses, where a blended learning solution (El-

Deghaidy and Nouby, 2008) was adopted. This

solution combined three ingredients: self-paced

learning (Ellis, 2007), live e-learning in a virtual

classroom (Stahl, 2005) and traditional classroom

learning (Michell, 2001). The aim behind this

teaching style, adopted in 2004 to 2005, was to bring

learner satisfaction back up to the level it reached

with the traditional classroom courses and keep the

costs as low as they were for distance learning

without the instructional model.

This paper presents the experience gained since

the late 1990s in teaching courses related to

information and communications technologies

across three different teaching modes: traditional

face-to-face classroom learning, distance learning of

virtualized educational contents without an

instructional model and distance learning using the

outlined instructional model. The results presented

here refer to learner satisfaction with each of the

three teaching modes for the particular case of a Java

programming course taught by the same teachers to

a set of learners chosen at random and divided into

three groups to receive instruction in one of the three

teaching modes.

2 THE INSTRUCTIONAL MODEL

The instructional model deployed in the third

teaching mode used for information and

communications technology courses is based on the

fact that teaching should enable learners to apply the

concepts that they learn to perform their jobs and

evaluate the outcomes. Learners must be motivated

to learn the educational contents covered in the e-

learning course with which they were previously

unfamiliar and be able to apply them to carry out

new tasks. Results will not always be as expected. In

this case, learners will have to review the decisions

taken and take corrective actions. This way, learners

will be able to internalize the knowledge that they

have learned.

The instructional model used is based on the

systematic development of instruction and learning.

It is composed of the following phases: analysis,

design, development and deployment, and execution

and evaluation.

2.1 Analysis

This phase defines what the course should teach.

The purpose of this phase is to find out what needs

future learners have in order to define appropriate

resources and analyse the best suited educational

contents. The results of this phase are the learning

objectives and the teaching contents making up the

course. The educational contents define what the

student should learn by performing a specified set of

tasks.

The educational contents are represented by a

knowledge graph. The knowledge graph nodes

CSEDU 2010 - 2nd International Conference on Computer Supported Education

142

represent the learning objectives, and the directed

lines connecting the nodes represent the tasks that

they have to carry out to reach a particular

knowledge state. This way, it is possible in this

phase to establish all the possible knowledge

sequences taking learners from an initial knowledge

state to the target knowledge state set for the course,

including all the tasks that learners have to complete

to able to do this.

2.2 Design

This phase defines how the learning process should

be carried out, that is, it specifies the learner’s

learning process, defining the learning approach, the

structure and depth of the concepts to be taught, the

process of executing the course and the expected

learner outcomes. Based on the knowledge graph

established in this last step, all the possible paths

leading from the initial knowledge state to the target

knowledge state are defined. This results in a

roadmap describing all the possible learning

processes required to achieve the course learning

objectives. Additionally, this design phase defines

the tasks learners have to complete to achieve each

of the learning objectives, group problem solving

and assessment exercises to check that learners have

acquired the concepts covered by the above learning

objectives.

2.3 Development and Deployment

The development of the course involves choosing

the best of all possible paths defined in the roadmap

developed in the last phase taking the learner from

the initial knowledge state to the target knowledge

state. This path includes a schedule of educational

contents, tasks to be completed and assessment

exercises. Additionally, deployment involves

implementing the course on a learning management

system platform.

2.4 Execution and Evaluation

This phase involves the learner using the learning

process. This instructional model takes a blended

learning approach to the learning process that

includes three learning types: self-paced learning,

live e-learning and face-to-face classrooms. Self-

paced learning is an asynchronous learning mode

available to learners anytime and anywhere.

Learners use digitalized and virtualized material

hosted by a LMS (learning management system),

including exercises and activities set for learners.

Learners complete self-assessments to evaluate what

they have learned before they are allowed to access

the next learning objectives. Learners are tutored

and can communicate with their peers and the

teacher over the Internet. Live e-learning is a

collaborative learning mode implemented through

videoconferencing, online chats, threaded

discussions or virtual classrooms scheduled at the

start of the course. Finally, face-to-face traditional

classrooms enable learners to get to know each other

and the teacher.

In our case study, the blended learning process

has been adapted to four-week courses with a total

student workload of 40 hours. The course starts with

a face-to-face classroom session where the professor

explains the aims of the course, the teaching

schedule and the exercises to be completed both as a

group and individually, as well as the assessment

exercises. Apart from enabling learners and teachers

to get to know each other, another aim of this

session is to form work groups. One-hour

interactions between learners and between learners

and the instructor are scheduled throughout the

course. They are held every three days via chat.

Computerized videoconferences are broadcast every

week. There is also permanent e-mail support, and,

finally, a face-to-face assessment is held

immediately after the course comes to an end.

During execution, information on the problems

encountered and the knowledge acquired is gathered

and logged to be analysed for monitoring purposes

to determine success and ascertain the learning

product quality.

3 STUDY DESIGN AND RESULTS

The goal of the study is to analyse the evolution of

learner satisfaction depending on each of the three

implemented learning modes. The results presented

in the paper are for a Java programming course. The

first learning mode is composed of eight five-hour

sessions taught in a face-to-face classroom. In the

case of distance teaching without an instructional

model, where the educational contents used in the

traditional classroom course were merely

virtualized, two face-to-face sessions were taught:

one at the start of the course to present the course

aims and teaching schedule and the other at the end

to hold the assessment examination. The other

distance teaching mode uses the described

instructional model combined with a blended

learning approach. In this case, two 45-minute face-

to-face sessions were held at the start and end of the

LEARNER SATISFACTION WHEN APPLYING AN INSTRUCTIONAL MODEL IN E-LEARNING - An Experimental

Study

143

course for the same purposes. Teachers provide

support through videoconferencing, interactive chat

sessions and permanent e-mail support.

To run the experiment, the same course was

taught by the same teachers in the three teaching

modes (independent variable) to a population of

different students in each case to stop the results

from being influenced by or depending on the

instructor teaching the course. To prevent the

outcomes being biased by the fact that the learners

enrolling for different teaching modes of the course

were not the same, the population was chosen to

assure that all the subjects perform similar jobs with

the same responsibilities and commitments.

Additionally, students took a level test at the start of

the course, and learners that attained a similar grade

were chosen for this study. A total of 225 learners

were involved in the study: 75 in each of the three

teaching modes. The criterion measured for the

dependent variable was the level of learner

satisfaction. This measurement was taken from a

questionnaire administered in the face-to-face

session at the end of the course in each learning

mode. This questionnaire contained three

statements: A1 “The course content meets my

training needs”, A2 “What I learned will be

applicable in my job” and A3 “The applied

methodology, technical resources and teaching

materials were appropriate”. The participants scored

their agreement or disagreement with each of the

above statements on a six-point Likert scale, ranging

from strongly disagree (scored as 1) to strongly

agree (scored as 6).

Table 1 shows descriptive statistics for the

learner satisfaction dependent variable: column 1

shows the three statements for all three executed

learning modes, column 2 indicates the size of

sample N and columns 3 and 4 list the mean and

standard deviation for the scores of the questionnaire

statements, respectively. Even though the sample

size was 75 learners for each of the three course

teaching modes, lower values in column N of Table

1 indicate that some learners failed to score the

statement specified in the respective table row.

Accordingly, 73 of the 75 learners participating in

the study scored questionnaire statement A2

administered at the end of the traditional classroom

course. Figure 1 plots similar information (mean and

standard deviation).

Table 1: Descriptive statistics for learner satisfaction

dependent variable.

Teaching method /

Statement

N Mean Std. Dev.

Traditional face-to-face

classroom:

A1 75 4.21 1.34

A2 73 4.43 1.32

A3 72 4.12 1.32

Distance learning without

instructional model:

A1 74 3.26 1.38

A2 73 3.44 1.35

A3 71 3.15 1.32

Distance learning with

instructional model:

A1 75 4.43 1.39

A2 75 4.72 1.41

A3 74 4.68 1.31

From the results shown in Table 1 and Figure 1,

it is clear that learner satisfaction effectively drops

sharply from traditional teaching in a face-to-face

classroom (with a mean of 4.25 out of 5 across the

scores for all three statements A1, A2 and A3) to

distance teaching by just virtualizing teaching

contents without taking into account an instructional

method (with an overall mean in this mode of 3.28).

To bring satisfaction back up to the level of the face-

to-face classroom, we used the instructional model

described in this paper. The outcome was a mean of

4.61 across scores for all three questionnaire

statements. From Table 1 and the chart in Figure 2,

we find that the mean values of the responses to the

three questionnaire statements are highest for the

instructional model. Even so, the score for A2 is

slightly higher, which means that the learning is of

practical use in the world of work. Also, the score of

4.43 for the case of A1, the statement most closely

related to satisfaction, indicated that teaching with

the instructional model was the one that best

satisfied learner needs.

Noteworthy, finally, is the fact that the standard

deviations across all the scores for statements A1,

A2 and A3 of the questionnaires range from 1.31

and 1.41. This means that there is a more or less

equal and low variance across the responses for each

statement regarding each of the three teaching

methods that we examined. This feature makes it

easier to compare the means statistically, and we can

say, without having to conduct an ANOVA, that the

mean scores for the statements on traditional

teaching and distance teaching with the instructional

model are statistically similar to each other and are

both greater than for distance teaching with

virtualized educational contents.

CSEDU 2010 - 2nd International Conference on Computer Supported Education

144

Figure 1: Mean and standard deviations for questionnaire

statement scores in each of the teaching modes.

We believe that these results can be explained as

follows. After years of teaching the Java

programming course in a face-to-face classroom,

experience was such as to achieve satisfactory

results in terms of learner satisfaction. Teaching this

course by merely digitalizing or virtualizing the

educational contents that were used in the face-to-

face classroom proved disastrous because it failed to

take into account even the most elementary

psychopedagogical prescriptions enabling the

adaptation of the teaching-learning process to the

Internet. Also a web-based instructional model is

required to guide the design and development of

both the contents to be taught and the actual

teaching/learning process. When these ingredients

were added to the distance education recipe, we

found that learner satisfaction again reached and

even rose above levels comparable to satisfaction

with face-to-face classroom teaching. We find then

that it is not practicable to migrate from traditional

education in the classroom to a new e-learning

paradigm by merely digitalizing and placing the

contents in a web server to make the resources

accessible anytime and anywhere.

4 CONCLUSIONS

This paper presents a study of satisfaction among

learners attending a Java programming course. Three

different teaching/learning modes have been used to

teach this course over recent years. First we used

traditional classroom sessions. Then, with the aim of

cutting costs, it was taught as distance learning. For

this purpose all the teaching contents were

virtualized. Finally, we included the described

instructional model. This model provides guidance

on course development, educational contents and

instruction. From the viewpoint of learning, a

blended learning solution was adopted combining

self-paced learning, live e-learning and face-to-face

classrooms.

The results of the study suggest that there is a

drop in learner satisfaction between learners

participating in face-to-face classroom sessions and

learners taking the web-based distance education

course with virtualized contents without an

instructional model. For this reason, we added the

instructional model described in this paper to

distance learning, and satisfaction moved back up to

what it had been originally. The study shows then

how the use of a proper distance teaching/learning

model leads to similar or better satisfaction levels

than traditional classroom sessions and has the plus

of cutting travel expenses. This is precisely why the

distance learning mode was introduced in the

information and communications technologies

courses. Note also that the effort and workload

required of teachers in this distance teaching mode is

much greater than in traditional classroom teaching,

which is to the benefit of learners. For this reason,

we have launched a pilot experiment deploying a

blended learning approach, similar to the one

described in this paper, in the learning process to

teach undergraduate students a subject that is part of

the undergraduate engineering degree in computing.

The goal of this experiment is to find out whether

the use of the innovative educational techniques

suited for distance education is able to reduce

underachievement in higher education or, at least,

increase the percentage of students that sit the final

subject examinations, that is, improve motivation.

ACKNOWLEDGEMENTS

This research was funded in part through educational

innovation projects related to e-learning

(IE08100581 and IE09100562) supported by the

Universidad Politécnica de Madrid.

REFERENCES

Altenhofen, M., Schaper, J., 2002. Flexible instructional

strategies for e-learning. In 35

Hawaii International

Conference on System Sciences, 401-410.

Anderson, N.H., 1996. A functional Theory of Cognition.

Lawrence Erlbaum Associates, New Jersey.

LEARNER SATISFACTION WHEN APPLYING AN INSTRUCTIONAL MODEL IN E-LEARNING - An Experimental

Study

145

Alonso, F., López, G., Manrique, D., Viñes, J.M., 2005.

An instructional model for web-based e-learning

education with a blended learning process approach.

British Journal of Educational Technology, 1, 217-

235.

Alonso, F., López, G., Manrique, D., Viñes, J.M., 2008.

Learning objects, learning objectives and learning

design. Innovations in Education and Teaching

International, 45, 4, 389-400.

Alonso F., Manrique, D., Viñes, J.M., 2009. A moderate

constructivist e-learning instructional model evaluated

on computer specialists. Computers & Education, 53,

57-65.

Arriaga, A., El Alami, M., Arriaga, F., 2003.

Neocampus2: New Trends in the Functionality of

Intelligent Learning Systems. In 3

International

Conference on Artificial Intelligence and Applications,

380-385.

Communication from the Commission to the Council and

the European Parliament, 2001. The e-Learning Action

Plan, Brussels, 28.3.

EL-Deghaidy, H., Nouby, A., 2008. Effectiveness of a

blended e-learning cooperative approach in Egyptian

teacher education programme. Computers &

Education, 51, 988–1006.

Ellis, H. J. C., 2007. An assessment of a self-directed

learning approach in a graduate web application

design and development course. IEEE Transactions on

Education, 50, 1, 55–60.

Govindasamy, T., 2002. Successful Implementation of e-

learning Pedagogical Considerations. Internet and

Higher Education, 4, 287-299.

Michell, L., 2001. E-Learning Methods Offer a

Personalized Approach. InfoWorld, 174-185.

Pazos, J., Azpiazu, J., Silva, A., Rodríguez-Patón, A.,

2002. A Virtual Classroom based on academic

memories. In Information Society and Education:

Monitoring a Revolution, 87-92.

Sampson, D., Karagiannidis, C., 2002. Accommodating

Learning Style in Adaptation Logics for Personalised

Learning Systems. In 14

World Conference on

Educational Multimedia, Hypermedia and

Telecommunications, Denver, Colorado, USA, 2002.

Stahl, G., 2005. Group cognition in computer-assisted

collaborative learning. Journal of Computer Assisted

Learning, 2, 79–90.

Tallent-Runnels, M.K., 2005. The Relationship between

Problems with Technology and Graduate Students’

Evaluations of Online Teaching. Internet and Higher

Education, 8, 167-174.

CSEDU 2010 - 2nd International Conference on Computer Supported Education

146