A Pedagogical-based Learning Object System to Support Self-regulated

Learning

Ali Alharbi, Frans Henskens and Michael Hannaford

School of Electrical Engineering and Computer Science, The University of Newcastle, Newcastle, Australia

Keywords:

Self-regulated Learning, Computer Science Education, Learning Styles, Learning Objects.

Abstract:

Self-regulated learning has become an important construct in education research in the last few years. Self-

regulated learning in its simple form is the learner’s ability to monitor and control the learning process. There

is increasing research in the literature on how to support students to become more self-regulated learners.

However, advancements in information technology have led to paradigm changes in the design and develop-

ment of educational content. The concept of learning object instructional technology has emerged as a result of

this shift in educational technology paradigms. This paper presents the results of a study that investigated the

potential educational effectiveness of a pedagogical-based learning object system to assist computer science

students. A prototype learning object system was developed based on the contemporary research on self-

regulated learning. The system was educationally evaluated in a quasi-experimental study over two semesters

in a core course on programming languages concepts. The evaluation revealed that a learning object system

that takes into consideration contemporary research on self-regulated learning can be an effective learning

environment to support computer science education.

1 INTRODUCTION

The aim of any instructional approach is to provide

students with high-quality learning material and ed-

ucational tools. In the last few years, the concept

of self-regulated learning has received increasing at-

tention in educational research, especially higher ed-

ucation research, because of its importance for aca-

demic success and lifelong learning (Dettori and Per-

sico, 2008). Self-regulated learning (SRL) focuses on

the learner as central in the learning process and on

the explicit use of a variety of learning strategies. It

is therefore reasonable to gain a greater understand-

ing of the learner, as a precursor to best integrating

aspects of self-regulated learning in the teaching and

learning process. We argue that the traditional vision

of designing and delivering learning material must be

altered to place greater emphasis on students’ pref-

erences and needs and increase students’ control and

monitoring of their self-regulated learning.

In accordance with the advancement of educa-

tional technology, the current trend in the instruc-

tional design of learning material is the use of dig-

ital educational resources that have pedagogical ob-

jectives as learning objects (Sosteric and Hesemeier,

2002). Learning objects are distributed via online dig-

ital libraries known as learning object repositories.

There is an increasing effort to develop standards and

speciﬁcations for these learning objects, but most of

this effort focuses on technical development and ig-

nore pedagogy or educational theories, particularly

learning styles and self-regulated learning.

Learning objects can improve the teaching and

learning of many disciplines. In particular, computer

science education has been criticised for a lack of

reference to pedagogical theories. The teaching and

learning of computer science concepts are challeng-

ing tasks for both teachers and students (Ben-Ari,

1998). This has been reﬂected in the low level of

retention and success among computer science stu-

dents (Biggers et al., 2008). Today, computer sci-

ence students have diverse backgrounds, experiences

and preferences. Computer science involves study-

ing dynamic and abstract concepts that are difﬁcult

for students to understand using traditional teach-

ing and learning methods. Computer science is a

rapidly changing area that is driven by new technolo-

gies rather than pedagogy (Holmboe et al., 2001).

Self-regulated learning behaviour is typical of com-

puter science students because they must learn differ-

ent concepts in a very short time to keep abreast of

the dynamic changes in the ﬁeld (Rodriguez-Cerezo

106

Alharbi A., Henskens F. and Hannaford M..

A Pedagogical-based Learning Object System to Support Self-regulated Learning.

DOI: 10.5220/0004383801060115

In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 106-115

ISBN: 978-989-8565-53-2

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

et al., 2011).

This paper addresses the challenges associated

with the design and use of learning objects to improve

the teaching and learning of computer science. A ped-

agogical framework is proposed to improve the design

and use of learning objects based on the concept of

self-regulated learning and students’ learning styles.

The framework is then used to develop and evaluate

an online learning object system for a core course on

programming languages.

2 SELF-REGULATED LEARNING

Self-regulated learning educational paradigms focus

on the role of the learner in the learning process, and

view the teacher as facilitative rather than dominant

over the learning process. Self-regulated learners are

active participants in the learning process who utilise

metacognitive, motivational and behavioural strate-

gies (Zimmerman, 1990). Metacognition refers to

the awareness and control of the cognition process

and includes processes, such as goal setting, planning

and self-evaluation to control and monitor the learn-

ing process (Pintrich, 2004).

Although various models have been developed to

illustrate the process of self-regulated learning, they

are all based on Zimmerman’s Cyclical Model of

Self-Regulated Learning (Zimmerman, 2000). Zim-

merman’s model views self-regulated learning as an

integration process between personal, behavioural

and environmental processes. According to this

model, self-regulated learning occurs via three cycli-

cal phases: forethought, performance control, and

self-reﬂection (Zimmerman, 2000). These phases are

cyclical; feedback from the previous phases is used to

adjust the next phase.

The forethought phase involves processes that oc-

cur prior to learning, including goal setting and strate-

gic planning. Goal setting is the process of determin-

ing the outcomes of the learning task. Strategic plan-

ning involves the selection of strategies that are suit-

able for performing the task. The activation of pre-

vious knowledge that is required to accomplish the

learning task is essential in this phase.

The performance phase involves processes that

occur during learning, such as self-control and self-

observation. Self-control involves actual learning

strategies that students use to manage the learning

material (e.g., reading, note-taking, critical think-

ing, help-seeking, etc.). Self-observation involves

metacognitive monitoring strategies that students may

use to track and evaluate their progress, such as self-

recording and self-questioning. Students employ the

technique of self-recording to record each learning ac-

tivity and its results. They utilise the strategy of self-

questioning or testing to assess their understanding of

the learning material by performing a test to evaluate

performance against a predeﬁned goal or standard.

Self-reﬂection involves processes that follow

learning, such as self-judgment and self-reaction.

These processes are closely associated with self-

observation. Self-judgment involves two sub-

processes, self-evaluation and causal attributions.

Self-evaluation is the comparison of individual per-

formance against predeﬁned goals. It also involves

comparisons with the performance of other students

in the same class. The result of self-evaluation is

linked to the causal attribution to determine the cause

of this result. For example, a student’s poor perfor-

mance can be attributed to bad strategy selection, in-

sufﬁcient effort, or limited abilities. Self-judgment

is linked to self-reaction. Self-reaction involves two

sub-processes, self-satisfaction and adaptive infer-

ences. Self-satisfaction is the learner’s perception

about his/her performance, i.e., whether the learner

is satisﬁed or disappointed. Based on this perception,

the learner employs adaptive inferences to determine

how to change the self-regulated learning process to

achieve a better result. Adaptive inferences involve

changing the goals deﬁned in the forethought phase,

or choosing other strategies to perform the task.

3 LEARNING STYLES

Learning is the process whereby individuals acquire

new knowledge. Research indicates that students tend

to gather and process information in different ways.

These differences are known as learning styles. Many

deﬁnitions of the term ’learning style’ can be found

in the literature. Keefe (Keefe, 1988) deﬁnes learn-

ing style as the characteristic cognitive, affective and

psychological behaviors that serve as relativity sta-

ble indicators of how learners perceive, interact with

and respond to the learning environment. This is one

of the most comprehensive deﬁnitions of the learning

style, and is adopted by the National Association of

Secondary School Principals.

3.1 Summary of Research on Learning

Styles

Education researchers agree there are different learn-

ing styles that must be accommodated to improve the

teaching and learning process. In addition, empiri-

cal studies on the implications of different learning

APedagogical-basedLearningObjectSystemtoSupportSelf-regulatedLearning

107

styles for students’ performance have found signiﬁ-

cant differences in the levels of academic achievement

of students with different learning style (Akdemir and

Koszalka, 2008). One explanation for this result is

that the learning materials favour speciﬁc learning

styles and ignore other styles.

There is current debate on how to integrate learn-

ing styles into curriculum design and teaching and

learning activities. Lack of empirical studies that

evaluate the effectiveness of learning styles-based in-

terventions has made it difﬁcult to generate recom-

mendations for teachers and curriculum designers.

The research on learning styles focuses primarily on

identiﬁcation of students’ learning styles, and how

this might affect their academic achievements. In ad-

dition, research on learning styles follows a track that

differs from that of other educational theories. The

role of learning styles in self-regulated learning has

not been investigated and appears to offer a potential

direction for future research.

The main hypothesis that dominates research on

learning styles is called the matching hypothesis

(Cofﬁeld et al., 2004). This hypothesis argues that

if a learner is presented with learning material that

is compatible with his/her own learning style, his/her

learning process improves. Further, teaching meth-

ods that are mismatched with the learner’s style might

lead to difﬁculties in learning. However, research on

how this hypothesis could be applied in context to im-

prove the teaching and learning process in many disci-

plines, including computer science, is scarce. Learn-

ing style awareness was proposed in response to criti-

cal reviews of learning style theories as an alternative

and promising hypothesis for future research on learn-

ing styles (Cofﬁeld, 2004). This hypothesis claims

that knowledge of learning styles should be used to

increase self-awareness, which leads to improvements

in the learning and teaching process. Learners who

become aware of their learning styles are more likely

to be aware of their strengths and weaknesses and,

therefore, have greater control of their learning pro-

cesses. In addition, teachers who are aware of the

diversity of learning styles amongst their students are

most likely to adopt teaching approaches that appeal

to different types of students. In this case, knowl-

edge about learning styles is used to enhance meta-

cognition, which is an important component in any

self-regulated learning model.

3.2 Felder-Silverman Learning Style

Model

Learning styles can be identiﬁed using different learn-

ing style models. There are many learning style mod-

els proposed in the literature. The FelderSilverman

Learning Style Model (Felder and Silverman, 1988)

is a well-known learning style model that is heavily

used to identify students’ learning styles in many dis-

ciplines, especially in science and engineering educa-

tion. The Felder Silverman Learning Style Model has

been adopted in this study as the basis for identifying

students’ learning styles due to many reasons, some

of them are the following:

• It covers more than one level of learning style,

thus provides categorisation of students’ learning

styles based on multiple dimensions.

• It has been used to investigate the learning styles

of engineering students, groups of learners similar

to the target population of this study (e.g., (Felder

and Brent, 2005)) .

• The instrument used by this model is reported to

have satisfactory level of reliability (Felder and

Spurlin, 2005).

The Index of Learning Styles (ILS) is the instru-

ment that is used to identify learning styles based on

this model. This model consists of four dimensions

(Felder and Silverman, 1988):

Perception (Sensing/Intuitive): this dimension de-

scribes the type of information an individual preferen-

tially perceives. Sensing learners prefer concrete con-

tents and facts, and are detail-oriented, whereas intu-

itive learners prefer abstract concepts, theories, and

mathematical formulas, and dislike details. Sensing

learners tend to solve problems using well-established

methods, and dislike complications. Intuitive learners

like innovations, new ideas of solving problems, and

dislike repetition.

Input (Visual/Verbal): this dimension describes the

type of presentation an individual prefers. Visual

learners prefer learning through visual media, such as

pictures, charts, and diagrams, whereas verbal learn-

ers prefer spoken or written materials and explana-

tions. Both types of learners learn better when the

material is delivered using visual, verbal, and written

forms.

Processing (Active/Reﬂective): this dimension de-

scribes how the learner processes information. Active

learners prefer learning in groups, and they tend to try

things out, whereas reﬂective learners prefer working

alone, and tend to think about how things work before

attempting them.

Understanding (Sequential/Global): this dimen-

sion describes how the learner progresses towards

understanding information. Sequential learners pre-

fer following a logical, step by step linear approach,

whereas global learners prefer absorbing the learning

materials randomly, in large jumps, without follow-

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

108

ing a step by step approach, until they grasp the full

picture. Global learners need to grasp the full pic-

ture before going into the details. Courses are nor-

mally taught according to a sequential presentation

format. Sequential learners can learn effectively un-

der this method of instruction.

4 LEARNING OBJECT SYSTEMS

Advances in information technology have led to a

paradigm shift in the way that people communicate

and learn. Consequently, the development and de-

livery of learning materials are changing. To reﬂect

this paradigm shift, a new instructional technology

called learning objects emerged as a next generation

technique for instructional design, due to its capacity

for reusability, adaptability and scalability (Hodgins,

2002).

Increased interest in the concept of learning ob-

jects has led to a number of deﬁnitions and terms to

describe the idea behind learning objects. Sosteric

and Hesemeier(Sosteric and Hesemeier, 2002) syn-

thesised several deﬁnitions and deﬁned a learning ob-

ject as ”a digital ﬁle (image, movie, etc.) intended to

be used for pedagogical purposes”. A learning object

can be published through a variety of methods. The

most formal method of publishing learning objects

is through learning object systems. A learning ob-

ject system is any online platform or environment that

is used to facilitate authoring, indexing, distributing,

and delivering of learning objects (Ritzhaupt, 2010).

A learning object system uses a database in which

learning objects are stored along with their metadata

to be shared. These databases are usually knows as

learning object repositories.

Learning objects can support students in their self-

regulated learning of, for example, computer science

if pedagogical foundations are considered during the

design and delivery of these learning objects. There is

a paucity of underlying theory that guides the design

and use of learning objects (Wiley, 2000). Moreover,

the delivery of learning objects in online learning ob-

ject systems does not follow a predeﬁned pedagogical

model based on the latest research in self-regulated

learning (Alharbi et al., 2011a).

5 PEDAGOGICAL-BASED

LEARNING OBJECT SYSTEM

Education research on learning styles, and that on

self-regulated learning, appears to be isolated from

one another. Self-regulated learning models that con-

sider the diversity of students’ learning styles have

the potential to provide a comprehensive understand-

ing of the learning process (Cassidy, 2011). This

leads us to return to the metacognitive component

of self-regulated learning, which concerns the impor-

tance of the learner’s awareness and ability to con-

trol his/her cognition process. According to this com-

ponent, learning styles can be used to improve the

metacognitive process, which in turn enhances stu-

dents’ motivation and learning. In this way, the future

research on self-regulated learning and learning styles

can interact to provide a basis for empirical studies

that can produce pedagogical recommendations for

teachers and instructional designers.

This study presented in this paper synthesises con-

temporary educational research to provide a greater

understanding of the theory of learning styles by plac-

ing it in the context of self-regulated learning mod-

els. The result of this synthesis is a pedagogical self-

regulated learning framework with learning style as

one of its main components (Alharbi et al., 2011b).

This framework can be used as the basis for improv-

ing learning and teaching in many disciplines. How-

ever, in the current research, the framework is applied

to improve the design of learning object instructional

technology in computer science education.

5.1 System Components

Based on the proposed pedagogical framework, the

current study develops and evaluates a learning object

system with self-regulated learning support. Figure 1

shows the main components of the proposed learning

object system.

5.1.1 Learning Object Repository

The learning object repository is responsible for stor-

ing different learning objects that are designed to

support students in learning about programming lan-

guages concepts. This repository stores all learning

objects along with their optional XML ﬁles that de-

scribe the structure of learning objects, and deﬁne the

animation inside the learning object. All the learning

objects are stored in the repository and tagged with

relevant metadata to make it easy for students to ﬁnd

them.

5.1.2 Learning Style Awareness Module

The objective of this module is to increase students’

awareness of their learning styles and their use of

self-regulated learning strategies. This module con-

sists of an initial assessment of students’ use of self-

APedagogical-basedLearningObjectSystemtoSupportSelf-regulatedLearning

109

Figure 1: Learning Object System Components

regulated learning strategies and the identiﬁcation

of students’ learning styles. A research instrument

is used to measure students’ self-regulated learning

strategies. The learning strategies are categorised

based on the research on self-regulated learning. The

Index of Learning Styles (ILS) is used in this module

to identify students’ preferred learning styles based on

Felder-Silverman learning style model (Section 3.2)

This model describes the learner’s preferences based

on four dimensions, Sensing-Intuitive, Visual-Verbal,

Active-Reﬂective and Sequential-Global. Upon ﬁrst

plete the learning style assessment. After completing

the assessment, the module evaluates the responses

and determines the student’s preferred learning style.

The module increases students’ awareness by provid-

ing the result, together with a description and recom-

mended learning strategies. Although students are

permitted to access all learning objects in the sys-

tem, the recommended learning strategies consider

the strengths and weaknesses of students’ preferred

learning styles.

5.1.3 Self-assessment and Misconceptions

Detection Module

This module is responsible for generating self-

assessment questions that help students detect their

misconceptions related to the programming languages

concepts. These self-assessments are associated with

the learning objects that are designed to help stu-

dents overcome these misconceptions. This mod-

ule is also responsible for recording each student’s

self-assessment results in the self-regulated learning

record. Each assessment exercise is linked to a spe-

ciﬁc misconception about programming languages

concepts. Learners are given instant feedback after

completing each assessment.

5.1.4 Self-reﬂection Module

Meta-cognition is the most important self-regulated

learning process that requires greater attention in on-

line educational environments. This module extracts

information from the analysis of students’ behaviours,

which is stored in the self-regulated learning record,

and uses this information to help students develop

self-reﬂection skills. To detect a speciﬁc misconcep-

tion, a number of questions are developed and inte-

grated into the self-assessment. The self-reﬂection

support module extracts information from the re-

sults of self-assessments that are stored in the self-

regulated learning record, and uses them to calcu-

late the degree of misconception related to a speciﬁc

concept. In addition, when a misconception is listed

in the student’s self-regulated learning record inter-

face, the module shows information on the propor-

tion of students with this misconception. This infor-

mation is shown to the leaner to encourage him/her

to overcome these misconceptions. The learner can

view additional information on the possible reasons

behind these misconceptions, and how to overcome

them by considering his/her learning styles. The mod-

ule also allows a student to view detailed information

on his/her behaviour inside the system, including the

time spent on each learning object compared to the

time spent by other students, and the results of the

self-assessment exercises.

5.1.5 Self-regulated Learning Record

The Self-Regulated Learning Record (SRLR) is a pro-

posed component that records the user’s interactions

with learning objects and other educational tools. The

self-regulated learning record provides an alternative

approach for the communication between LMS and

different types of learning objects. The content of the

self-regulated learning record can be accessed by any

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

110

LMS or educational tool, which in turn supports self-

regulated learning. In the proposed online learning

object system, the SRLR stores information related to

the learner and his/her use of learning objects. This

information includes the following:

• Time student spent on each learning object per

session..

• The results of students’ learning styles and learn-

ing strategies assessments.

• The results of students’ self-assessments.

• Students’ navigation behaviour in each session.

6 EVALUATION

METHODOLOGY

This section presents details of the research method-

ology that was adopted to evaluate the educational ef-

fectiveness of the proposed learning object system.

These details include a description of the research

participants, design, and procedure. In addition, the

instruments that were used to collect the data are de-

scribed. This section concludes by describing the data

analysis techniques used to analyse the data and how

the results were interpreted to test the hypotheses and

answer the research questions.

6.1 Research Design:

Quasi-experimental Study

The qualitative portion of the study follows a quasi-

experimental control group design with pre-tests and

post-tests (Creswell, 2012). Quasi-experimental de-

sign is the same as the control experimental design

except that the participants are not randomly assigned

to the experimental conditions. Rather, intact con-

venience groups are used. This design is commonly

used in educational research due to difﬁculties associ-

ated with randomly dividing participants into groups.

The purpose of the experiment is to determine the ef-

fect of the proposed educational intervention on stu-

dents’ academic achievement in a core computer sci-

ence course.

6.2 Description of the Course and

Participants

The participants in this study are students enrolled

in the programming languages and paradigms course

at the University of Newcastle, Australia, in the ﬁrst

semesters of 2011 and 2012 respectively. The over-

all sample size was 62 students: 34 in 2011 (control

group) and 28 in 2012 (experimental group).

The online learning object system is used and

evaluated in the course Programming Languages and

Paradigms. A course that covers programming lan-

guage concepts is important for computer science and

software engineering students and such a course is

an integral part of any computer science and soft-

ware engineering program (IEEE/ACM, 2005). Pro-

gramming language concepts are presented by com-

paring the features of programming languages, such

as Java and C++. In addition, several program-

ming paradigms are discussed and compared. The

Programming Languages and Paradigms course at

the University of Newcastle is a compulsory second

year course for undergraduate students enrolled in the

computer science and software engineering programs.

The course follows a traditional teaching method that

consists of weekly lectures and workshops.

6.3 Data Collection Instruments

A number of data collection instruments were utilised

to address the research the following research ques-

tions:

• What is the effect of the proposed learning object

system on students’ academic achievement?

• To what extent are students satisﬁed with the edu-

cational effectiveness of the system?

The following instruments were used to collect

both quantitative and qualitative data.

6.3.1 Students’ pre- and post-tests

Students tool the pre-test in both the experimental and

control groups at the beginning of the semester, and

before the experimental group was introduced to the

online learning object system. The pre-test consists

of questions to help students refresh their knowledge

about several object-oriented and data structures con-

cepts, and how to apply them to solve a real-world

problem.

6.3.2 Students’ Satisfaction Questionnaire

This instrument is an online questionnaire completed

by students to evaluate the educational effectiveness

of the entire learning object system at the end of the

semester. This instrument includes questions about

students’ perceptions of the educational effectiveness

of the online learning object system. The question-

naire utilises a 7-point Likert scale, with 1 repre-

senting strongly disagree and 7 representing strongly

APedagogical-basedLearningObjectSystemtoSupportSelf-regulatedLearning

111

agree. The questionnaire consists of dimensions that

are related to a speciﬁc feature of the online learning

object system. Each dimension has a number of ques-

tions.

6.3.3 Self-regulated Learning Record

Self-Regulated Learning Record (SRLR) is a compo-

nent of the online learning object system that auto-

matically logs all activities performed by students in

each session. This includes the frequency and time

spent on learning objects and the results of students’

self-assessments. Also, the students’ navigation be-

haviour in each session can be discovered by extract-

ing the information stored in the SRLR. The data col-

lected using the SRLR was used to study students’

behaviour inside the system.

6.4 Method and Procedure

The study was conducted in two consecutive phases.

In the ﬁrst phase (ﬁrst semester 2011), the control

group did not receive intervention and were taught

using the traditional instructional approach. In the

ﬁrst week, students were given an information state-

ment that described the research objectives and in-

vited them to participate. Those who agreed to par-

ticipate signed a consent form that indicated that they

were willing to participate in the research study as de-

scribed in the information statement. Then, the Index

of Learning Style (ILS) was administered to students

who signed the consent form to identify their pre-

ferred learning styles. In addition, the Self-Regulated

Learning Strategies Questionnaire was administered

to students to measure the level of use of different

self-regulated learning strategies. In the second phase

(ﬁrst semester 2012), the experimental group received

the online learning object system as an educational

intervention to aid in developing self-regulated learn-

ing strategies while studying the course material. The

Index of Learning Style (ILS) was given to the stu-

dents at the beginning of this phase using the same

questionnaires that were used in ﬁrst phase. How-

ever, in the second phase, additional research instru-

ments were used to measure the educational effective-

ness of different aspects of the research intervention.

These include the students’ satisfaction questionnaire

and the self-regulated learning record.

7 RESULTS AND DISCUSSIONS

7.1 The Result of the

Quasi-experimental Study

The ﬁrst step to evaluate the educational effectiveness

of the intervention is to report descriptive statistics

that describe the academic performance of the stu-

dents in the control and the experimental groups. The

second step is to perform hypothesis testing. This is

the formal procedure used by statisticians to accept or

reject hypotheses. The statistical level of signiﬁcance

(α) is set to 0.05 for hypothesis testing. The anal-

ysis also evaluates the inﬂuence of students’ learn-

ing styles and level of self-regulated learning on their

academic performance in both groups. An analy-

sis of students’ behaviour in the online learning ob-

ject system was also conducted. An analysis of co-

variance (ANCOVA) was used to measure the differ-

ence between the control and the experimental groups

while taking into consideration the possibility of pre-

existing differences between the two groups.

In the control group, the mean ﬁnal exam score

was 55.2, while it was 65.3 in the experimental group.

To test whether this difference is statistically signiﬁ-

cant, a one-way analysis of variance (ANCOVA) was

used. The following hypotheses were formulated:

H0: there is no signiﬁcant difference in the ﬁnal

exam scores between the control and the experimental

group.

HA: there is a signiﬁcant difference in the ﬁnal

exam scores between the control and the experimental

group.

The independent variable is the medium of in-

struction, which consists of two levels, traditional

or intervention. The dependent variable is students’

achievement scores on the ﬁnal exam. Students’

scores on the pre-test were considered as the covari-

ate in the ANCOVA to control for the pre-existing

differences between the control and the experimental

groups (Table 1).

Table 1: The result of the statistical test for the ﬁnal exam

scores comparison.

Group N M SD F P

Control 34 55.2 15.9

9.83 0.003*

Experimental 28 65.3 19.5

The result of the ANCOVA was signiﬁcant (F

=9.83, p=0.003 <0.05). Based on this result, the

null hypothesis (H0) was rejected and we accepted

the alternative hypothesis (HA) that the difference in

the mean ﬁnal exam scores between the experimen-

tal and control groups is statistically signiﬁcant. Stu-

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

112

dents in the experimental group (M=65.3, SD=19.5)

signiﬁcantly outperformed those in the control group

(M=55.2, SD=15.9) on the ﬁnal exam after consider-

ing the pre-test scores as a baseline for both groups.

Thus, regardless of the pre-existing difference in stu-

dents’ achievement on the pre-test between the study

groups, the online learning object system had a sta-

tistically signiﬁcant positive effect on the ﬁnal exam

scores of the experimental group.

7.2 Students’ Satisfaction

At the end of the semester, the ﬁnal feedback ques-

tionnaire was made available online inside the learn-

ing object system. This questionnaire measured stu-

dents’ degree of satisfaction with the learning object

system. The satisfaction questionnaire consists of a

number of dimensions; each measures students’ sat-

isfaction in terms of their perceptions about a speciﬁc

feature of the online learning object system Nineteen

(experimental group) students completed the ques-

tionnaire at the end of the course. This section sum-

marises the analysis of students’ responses to the

questions related to each dimension in the question-

naire. Each dimension consists of a number of ques-

tions. Students responded using a 7-point Likert

scale ranging from 1 (strongly disagree) to 7 (strongly

agree). The average percentage of students who re-

sponded to each level of the Likert scale was calcu-

lated for each dimension.

The ﬁrst dimension of the questionnaire measured

students’ satisfaction with the online learning object

system in terms of their perceptions about the abil-

ity of the system to correctly identify their preferred

learning style, and the system’s recommendations and

guidelines. The result is presented in Figure 2. The

majority of students (93%) had a positive percep-

tion of the learning style identiﬁcation and awareness

module. They considered it to be useful in providing

recommendations and guidelines that reﬂected their

preferred learning styles. The majority of students

agreed that the system helped them to identify their

preferred learning styles and that the learning strate-

gies were easy to follow and useful. The result indi-

cates that the system helped students gain awareness

of their learning styles. Thus, many of them will be

aware of their learning styles in their future studies,

and should be able to use this knowledge to aid their

self-regulated learning, utilising the strengths of their

learning styles and overcoming their weaknesses.

The second dimension of the questionnaire mea-

sured students’ satisfaction in terms of perceptions

about the self-reﬂection support module which was

used in the system to help students monitor and

Figure 2: Students’ satisfaction with the learning style

awareness module.

Figure 3: Students’ satisfaction with the self-reﬂection

module.

control the self-regulated learning process. Figure

3 shows that 72% of students agreed that the self-

reﬂection support module, which was used to record

students’ interactions with learning objects, informa-

tion related to their misconceptions, and indicators of

their progress, was educationally effective. Of the stu-

dents, 16% had a neutral opinion and 12% disagreed

with the educational beneﬁt of the self-reﬂection sup-

port module.

The last dimension measures students’ overall

perceptions of the idea of using online learning ob-

ject systems to support self-regulated learning. The

result is presented in Figure 4. The ﬁnal result of

the questionnaire indicates that 94% of the students

agreed with the statements comprising this dimen-

sion. Nearly all students strongly supported the idea

of applying online learning object systems to other

computer science courses.

APedagogical-basedLearningObjectSystemtoSupportSelf-regulatedLearning

113

Figure 4: Students’ satisfaction with the idea of learning

object systems.

7.3 Analysis of Navigation Behaviour

Trace analysis of students’ navigation behaviour was

conducted to study students’ behaviour inside the

system, using the information recorded in the self-

regulated learning record. Based on the self-regulated

learning model used in this study, we proposed a navi-

gation behaviour analysis method to classify learners’

self-regulated learning behaviour. A number of nav-

igation behaviour patterns were observed. We con-

ducted further analysis only on the patterns that were

followed frequently. These patterns were categorised

as follows:

Browsing: this behaviour implies that students jump

between different pages inside the system in the same

session without spending more time on the learning

objects or their self-regulated learning record.

Unplanned View of Learning Objects: this be-

haviour implies that students view learning objects

that are most likely not related in the same session.

Students who adopted this behaviour typically did not

complete a self-assessment after viewing the learning

object.

Inefﬁcient Use of Self-assessments: this behaviour

implies that the student tends to take self-assessments

for different topics in the same session without or

with a limited view of learning objects and their self-

regulated learning record inside the system.

High Level of Meta-cognition: this behaviour im-

plies that students tend to follow a navigation path that

is consistent with the self-regulated learning model.

They tend to view their self-regulated learning record

at the beginning of each session, then view learn-

ing objects related to one topic only, complete self-

assessments and spend time reading the feedback af-

ter submitting the self-assessment. They also tend

to make decisions based on their results in the self-

assessments, such as viewing learning objects again

and then completing the self-assessment again.

Table 2: Navigation behavior patterns.

Navigation pattern Proportion

of stu-

dents

Browsing 14%

Unplanned view of learning objects 14%

Inefﬁcient use of self-assessments 18%

High level of meta-cognition 54%

The proportion of students who frequently

adopted each navigation pattern is presented in Ta-

ble 2. More than half of the students (54%) adopted

a behaviour pattern that reﬂects a high level of meta-

cognition inside the online learning object system. Of

the students, 18% showed a tendency to adopt naviga-

tion behaviour that reﬂected inefﬁcient use of the self-

assessment exercises and 14% frequently adopted a

behaviour that reﬂected browsing behaviour or un-

planned view of learning objects.

Further analysis of the data was conducted to in-

vestigate the inﬂuence of the most frequently adopted

navigation behaviour pattern on students’ academic

achievement as measured by the post-test. To fa-

cilitate the analysis, the three groups who did not

adopt the meta-cognition pattern frequently were

combined together to form one single group (non

meta-cognition). After that, an ANCOVA test was

conducted to test if there is any signiﬁcant difference

in the post-test scores between students in the meta-

cognition pattern and the non-meta cognition pattern

groups, after controlling for their pre-test scores. Ta-

ble 3 presents the result of the ANCOVA statistical

test.

Table 3: Navigation patterns.

Navigation pattern Mean SD F P

Meta-cognition 74.9 12.2

7.68 0.010*

Non meta-cognition 54.3 20.9

Table 3 shows that the meta-cognition be-

haviour pattern group had higher post-test scores

(M=74.9, SD=12.2) than the non meta-cognition

group (M=54.3, SD=20.9). The result of the AN-

COVA test conﬁrms that this difference is statistically

signiﬁcant after adjusting for the pre-test scores, (F

=7.68, p=0.010 <0.05). To sum up, students who

adopted the meta-cognition behaviour pattern more

frequently had higher academic achievements as mea-

sured by the post-test.

8 CONCLUSIONS

This paper presented the result of an empirical study

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

114

that evaluated the educational effectiveness of an on-

line learning object system to support self-regulated

learning of programming languages concepts. The

system design was based on a pedagogical frame-

work that was adopted to improve the role and im-

pact of learning object repositories. The result of the

study revealed that the learning object system is an

effective intervention in supporting students as self-

regulated learners. This was also reﬂected in the re-

sults of the students’ satisfaction questionnaire, which

showed that the students had positive perceptions of

the features of the system.

REFERENCES

Akdemir, O. and Koszalka, T. (2008). Investigating the rela-

tionships among instructional strategies and learning

styles in online environments. Computers & Educa-

tion, 50(4):1451–1461.

Alharbi, A., Henskens, F., and Hannaford, M. (2011a).

Computer science learning objects. In e-Education,

Entertainment and e-Management (ICEEE), 2011 In-

ternational Conference, pages 326–328, Jakarta, In-

donesia. IEEE.

Alharbi, A., Henskens, F., and Hannaford, M. (2011b). Ped-

agogical framework to improve the quality of learning

objects in collaborative e-learning systems. In 3rd In-

ternational Conference on Computer Supported Edu-

cation, pages 353–358, Valencia, Spain.

Ben-Ari, M. (1998). Constructivism in computer science

education. SIGCSE Bull., 30(1):257–261.

Biggers, M., Brauer, A., and Yilmaz, T. (2008). Student

perceptions of computer science: a retention study

comparing graduating seniors with cs leavers. ACM

SIGCSE Bulletin, 40(1):402–406.

Cassidy, S. (2011). Self-regulated learning in higher educa-

tion: Identifying key component processes. Studies in

Higher Education, 36(8):989–1000.

Cofﬁeld, F. (2004). Learning styles and pedagogy in post-

16 learning: A systematic and critical review.

Cofﬁeld, F., Moseley, D., Hall, E., and Ecclestone, K.

(2004). Should we be using learning styles?: What

research has to say to practice. Learning and Skills

Research Centre London.

Creswell (2012). Educational Research: Planning, Con-

ducting, And Evaluating Quantitative And Qualitative

Research. Pearson, 4th edition.

Dettori, G. and Persico, D. (2008). Detecting self-regulated

learning in online communities by means of interac-

tion analysis. Learning Technologies, IEEE Transac-

tions on, 1(1):11–19.

Felder, R. M. and Brent, R. (2005). Understanding stu-

dent differences. Journal of Engineering Education,

94(1):57–72.

Felder, R. M. and Silverman, L. K. (1988). Learning and

teaching styles in engineering education. Engineering

education, 78(7):674–681.

Felder, R. M. and Spurlin, J. (2005). Applications, reliabil-

ity and validity of the index of learning styles. Interna-

tional Journal of Engineering Education, 21(1):103–

112.

Hodgins, W. (2002). Learnativity: into the future. E-

learning Handbook, McGraw Hill, New York, pages

38–64.

Holmboe, C., McIver, L., and George, C. (2001). Re-

search agenda for computer science education. In

Proceedings of the 13th Workshop of the Psychol-

ogy of Programming Interest Group, pages 207–233.

Bournemouth University, London, England.

IEEE/ACM (2005). Computing curricula 2005, an overview

report.

Keefe, J. (1988). Proﬁling & Utilizing Learning Style.

ERIC.

Pintrich, P. (2004). A conceptual framework for assess-

ing motivation and self-regulated learning in college

students. Educational psychology review, 16(4):385–

407.

Ritzhaupt, A. D. (2010). Learning object systems and

strategy: A description and discussion. Interdisci-

plinary Journal of E-Learning and Learning Objects

(IJELLO), 6:217–238.

Rodriguez-Cerezo, D., Gomez-Albarran, M., and Sierra,

J. L. (2011). Supporting self-regulated learning in

technical domains with repositories of learning ob-

jects and recommender systems. In Advanced Learn-

ing Technologies (ICALT), 2011 11th IEEE Interna-

tional Conference on, pages 613–614. IEEE.

Sosteric, M. and Hesemeier, S. (2002). When is a learn-

ing object not an object: A ﬁrst step towards a theory

of learning objects. The International Review of Re-

search in Open and Distance Learning, 3(2).

Wiley, D. (2000). Connecting learning objects to instruc-

tional design theory: A deﬁnition, a metaphor, and a

taxonomy. Learning Technology, 2830(435):1–35.

Zimmerman, B. (1990). Self-regulated learning and aca-

demic achievement: An overview. Educational psy-

chologist, 25(1):3–17.

Zimmerman, B. (2000). Attaining self-regulation: A social

cognitive perspective. Handbook of self-regulation,

13:14.

APedagogical-basedLearningObjectSystemtoSupportSelf-regulatedLearning

115