QUALITY CONTENT MANAGEMENT FOR E-LEARNING

General issues for a decision support system

Erla Morales

Departament of Theory and History of Education, Salamanca University, Canalejas Street 169, Salamanca, Spain

Francisco García

Department of Computer Science, Salamanca University, Merced Square, s/n, Salamanca,Spain

Keywords: Knowledge management, decisio

n support systems, learning objects

Abstract: In today’s world, reusable learning object concepts and standards for their treatment represent an advantage

for knowledge management systems to whatever kind of business that supports an on-line system. Users are

able to manage and reuse content according to their needs without interoperability problems. The possibility

of importing learning objects for e-learning aim to increase their information repository but the learning

object quality is not guaranteed. This work proposes a system to manage quality learning objects to support

teachers to select the best content to structure their course. To achieve this we suggest two subsystems:

First, an importation, normalization and evaluation subsystem; and second, a selection, delivery and post

evaluation subsystem.

1 INTRODUCTION

Through an e-learning repository we can find a

myriad of content from academic research and

contributions, but how about the content’s quality?

Without doubt, an important contribution from

com

puter science to knowledge management and e-

learning systems is the learning object (LO) concept.

This element has characteristics of independent

units, which are able to be reused for other

educational situations. According to this, knowledge

management for e-learning based on reusable LOs

means the possibility to access specific content

according to the learners’ needs.

The stage mentioned above is possible due to

andards, which were established as an attempt to

avoid interoperability platform problems. Thanks to

reusable LOs and standards for their treatment,

knowledge management becomes more easy and

efficient but it doesn’t guarantee the content quality.

A great quantity of criteria exists about digital

earning sources evaluation. Nevertheless, for LO

content evaluation there are just a few proposals

which consider their characteristics. So it is

necessary for a knowledge management system to

frequently re-feed the content for an e-learning

repository together with the teacher’s expert

knowledge and the student’s learning experience. In

this case we are focused on teacher’s expert

knowledge.

On this basis, section 2, presents general issues

r LOs management. Section 3 presents the main

elements for systems to support decisions.

Subsection 3.1, presents a subsystem to import LOs

from external sources, followed by our

recommendation to normalize them according to a

knowledge model and finally to evaluate them

through an instrument and collaborative strategy.

Subsection 3.2 explains another subsystem that

supports decisions and content re-feed. Finally,

section 4 summarizes conclusions and further work.

2 LEARNING OBJECTS

MANAGEMENT

The knowledge management we suggest is based on

reusable LOs. The most widespread definition is

from (IEEE LOM, 2002) that states the “digital or

non-digital entity that may be used, reused or

referenced while the learning receives technical

support”

343

Morales E. and García F. (2005).

QUALITY CONTENT MANAGEMENT FOR E-LEARNING - General issues for a decision support system.

In Proceedings of the Seventh International Conference on Enterprise Information Systems, pages 343-346

DOI: 10.5220/0002524003430346

 SciTePress

However, this concept is too broad to guarantee

an efficient resources management. We believe LOs

should represent at least a single instructional

objective and all of the related materials required to

support that objective

For this reason we propose a definition adapted

by (Polsani, 2003) “A Learning Object is an

independent and self-standing unit of learning

content that is predisposed to reuse in multiple

instructional contexts”, on this basis we refer to

learning objects which contain a unit of learning

with educational sense, for example a lesson.

3 A SYSTEM TO SUPPORT

DECISION

According to LOs and standards capabilities, it is

necessary to consider how to manage quality LOs,

taking into account their characteristics to help

teachers to structure their courses. For this reason we

suggest a system to support decisions about how to

select the best content from a LOs repository. Figure

1 presents a general view about the system we

propose which will be explained in the next sections.

3.1 Importation, normalization and

evaluation subsystem

A first step we must consider is to import LOs

content to the e-learning repository. According to

this, for knowledge management it is necessary to

take into account some questions about content such

as what, how and why should it be managed

(Kuang-Tsae, 2000). Taking these issues into

consideration, imported LOs may be selected with

regard to context issues (Marquès, 2001). On this

basis, some keywords may be used for searching a

suitable LO.

After that, the imported LOs can be saved into a

non–normalized content repository, because each

one of them could has a different granularity level

than others ones. Therefore, to import LOs from

external sources the second step we suggest is to

normalize imported LO’s according to a knowledge

model. To achieve this, we suggest the next steps to

normalize LO’s.

1.- Clasiffy LO’s objectives according to their

complexity level, because this way it is easier

knowing if the LO is suitable for new educational

situations. Then we suggest Bloom’s cognitive

domain taxonomy (Bloom, 1956) because it has

been widely used in e-learning to define cognitive

objectives and also it divides the objectives into high

and low complexity levels.

2.- Define the difficulty level to each one of LO, for

this ussue we propose three kinds of complexity

levels: basic, medium and advanced because this

kind of clasiffication would help teachers to select

the LO content according to their teaching

objectives.

3.- Classify the imported LO into three kind of

content areas: data and concept, procedure or

processes, and reflection or attitude. This

classification aims to define the kind of content

according to the learning objectives.

4.- Classify the imported LO into three kind of

activities: Initiation, Re-structuring and Application.

Initiation activities classification may be for all LOs,

which are designed to teach basic content for a

specific subject. Restructuring activities

classification may be directed to promote new

knowledge acquisition. Finally, applying

classification activities may be directed to promote

students’ experience in order to achieve their new

Knowled

e mana

ement

Teacher(s) Experts Teacher(s) Students

1.Import UL

2. Normalization

(knowledge model)

Non normalized

repository

3. Evaluation

(instrument)

4. Selection

(quality LO’s)

5.Delivery courses

(e-learning system)

6.Student’s evaluation

(experience)

Normalized

and evaluated

subsystem

Feedbac

Imporation,

normalization, and

evaluation subsystem

Selection,

delivery and post

evaluation subsystem

ure 1: A decision su

ort s

ste

ICEIS 2005 - ARTIFICIAL INTELLIGENCE AND DECISION SUPPORT SYSTEMS

344

concepts acquisition.

Nevertheless, the classification of the LO

according to a knowledge model like this is not

enough to guarantee the LO quality.

There exists is a plethora of quality criteria to

value digital sources but there are only a few

proposals about how to evaluate LOs

In order to achieve an optimal evaluation of the

LOs, it is necessary on one hand considering quality

criteria from different kinds of categories to each

one of LO, and on the other hand, considering the

LOs evaluation models (Merlot, 2003; Vargo et al.,

2003, Williams, 2000). In this way, it is possible to

consider different points of view with regard to the

same object. According to this we suggest a third

step for knowledge management to support

decisions, it is an instrument which considers

different evaluation criteria in four categories.

Psychopedagogical category (30%): This kind

of criteria aims to determine if the LO is suitable to

promote learning, for example, learner’s motivation.

Didactic-curricular category (30%): This kind

of criteria aims to evaluate if an object is related to

curricular objectives according to the context in

which it will be applied.

Technical- aesthetic category (20%):

Technical-aesthetic criteria aim to evaluate issues

like legibility, color-contrast, etc.

Functional category (20%): It aim to evaluate

if an object work correctly and doesn’t obstruct the

learning process.

From the stages mentioned above, the

psychopedagogical and didactic-curricular

categories are more important than technical-

aesthetics and functional categories within the

educational context, then, we do not propose

evaluating them with the same score weighting. We

suggest evaluating each object with the same rating

scale but applying a different percent.

For getting the final result, we propose

calculating the average score gained for each object

according to the percent weighted for each category

with the following rating scale: 0 = Criteria is not

present; 1 = Very low; 2 = Low; 3 = Medium, 4 =

High, 5 = Very high.

Due to the fact that an optimal LO evaluation

considers criteria from different kinds of categories,

we suggest the participation of different kinds of

experts during the evaluation, for example:

instructional designers, subject experts, and so on.

The participation of at least one participant from

each area encourages not only different points of

view over the subject under evaluation, but also a

critical objectivity and a reliable LO evaluation.

We propose two modes of applying the

instrument suggested above in order to value the

LO: individual and collaborative method.

According to this concept, individual evaluation

provides us with an initial appreciation of the quality

of the LO based on the judgment of each participant.

For making easier this evaluation firstly we

propose the possibility to view the LO Metadata

(IMS LOM, 2003) through the e-learning platform.

It allows to the evaluators knowing quickly LOs

characteristics. After, we propose that the evaluators

may view all the evaluation indicators classified into

each category. It allows that the evaluators may

know the meaning of the criteria that they are

testing.

For the evaluation of LOs characteristics we

suggest two criteria. The first one is LO reusability,

which means assessing whether the LO can be

reused for other educational situations (into didactic-

curricular category). The second one is ensuring

standard compliance (into technical-aesthetic

category).

The possibility of completing an evaluation

through collaborative method enables one to contrast

the individual’s initial evaluation with the others

experts’ evaluations. It aims to share different points

of view to achieve an advanced and reliable

evaluation (Vargo et al., 2003). However, the

emergence of consensus is not always a fact, so we

suggest publishing evaluators’ disagreements

through the platform, and as a result it will be

possible to consider this information before the LO

is reused.

3.2 Selection, delivery and post-

evaluation subsystem

Once LOs evaluations are completed they will be

saved on a normalized repository, as shown between

three and four steps in Figure 1. This repository will

be required for teachers to search the content they

need to structure their courses, and from this

repository teachers can find quality and uniform

LOs.

Numerical ratings provided through the

evaluations mentioned above allow quick

comparisons for searching LOs.

LOs classifications provided for the knowledge

model and their evaluation allow teachers to find

content according to the subject area, type of

content, type of activity, and level of difficulty

(retrieving content associated with Bloom’s

cognitive domain categories) and their numerical

evaluation, which reflect their quality.

To achieve an optimal LO selection for reuse, we

suggest a knowledge management system with the

possibility to view a list of all the final LOs

evaluations and the possibility to access evaluation

criteria by links. As a result, it becomes easier to

QUALITY CONTENT MANAGEMENT FOR E-LEARNING: General issues for a decision support system

345

recognize which elements of the LO are weak and

find a way to correct, improve or change them.

LOs needs to be enabled with other ones to build

the largest units (didactic units, courses, etc.)

possible to deliver selected LOs for students, such as

those shown in step number five in Figure 1.

To achieve this objective, an educational

modeling language is needed. We also suggest IMS

Learning Design (IMS LD, 2003) because it has a

flexible structure that supports pedagogical

diversity. The classification provided by the

knowledge model could help for this work.

However, the LOs evaluation we suggested is not

definitive. Once the LO evaluation has ended, it is

necessary to make a LO re-evaluation, which

considers a learners’ experience about the efficacy

of the LO to improve its quality as shows six steps in

Figure 1. Therefore a re-feeding process begins

taking into account students’ and teachers’

contributions to the LOs quality. As Figure 1 shows,

the re-feeding process is a cycle in which content is

constantly evaluated for all the e-learning users.

4 CONCLUSIONS

We think the general issues discussed here have

important advantages. Nowadays, the LO concept is

widely discussed, however we are suggesting and

specific definition to their evaluation and

management for e-learning systems. In this way it is

possible to define what criteria and what quality

indicators we could use to evaluate them.

On other side, due to the different kind of LOs

definitions, a lot of LOs with different levels of

granularity exists. A knowledge model, like ours,

aims to normalize the imported LOs. In this way, it

is possible to manage uniform LOs for their

evaluation and classified them according to an

educational context.

The type of evaluation we are suggesting has

several advantages in comparison with other

proposals. There are few ways about how to evaluate

learning object, for example, MERLOT (2003)

proposes an evaluation with stars from 1 to 5,

considering just few evaluation criteria. However,

we propose an evaluation that involves different

kind of evaluation categories to get an integral

evaluation adding the possibility for evaluators to

view evaluation indicators to guide them.

Additionally, the proposed knowledge

management system could be an important

contribution for e-leaning systems. Educators could

make use of the information already existing and use

the information that most interests them to structure

their courses. Also, this proposal would help to

promote a more in-depth reflection and evaluation of

the syllabus by taking into account points of view

related to searching and utilizing quality educational

sources.

As a result, this proposal also could help students

make use of quality content and activities by taking

into account a variety of educational, curricular,

technical and functional points of view.

These would in turn guarantee the establishment

of an up-to-date knowledge-base that would be both

suitable and reliable in accordance with the needs

and requirements of learners.

In addition, feedback would assist in answering

the questions about how to manage a growing e-

learning information repository to meet the users’

needs. Our future work is to implement this model in

order to make possible adjustments and

modifications.

REFERENCES

Bloom, B., 1956. Taxonomy of Educational Objectives:

Handbook I, Cognitive Domain. David McKay.

IEEE LOM., 2002. IEEE Learning Object Metadata.

http://ltcs.ieee.org/wg12.

IMS LOM., 2003. Learning Resource Metadata

Specification.

http://www.imsglobal.org/metadata/index.cfm.

IMS LD., 2003. IMS Learning Design Specification.

http://www.imsglobal.org/learningdesign/index.cfm.

Kuang-Tsae, H., Lee, Yang, W., Wang, Richard., 2000.

Calidad de la información y gestión del conocimiento.

Editorial AENOR, Madrid.

Marquès, P.,

2001. Orientaciones para la selección de

materiales didácticos. http://dewey.uab.es/pmarques.

MERLOT., 2003. Multimedia Educational Resources for

Learning and Online Teaching. http://merlot.org.

Polsani, P., 2003. Use and abuse of reusable learning

objects. Journal of digital information.

http://jodi.ecs.soton.ac.uk/Articles/v03/i04/Polsani/

Vargo, J., Nesbit, J.C., Belfer, K., Archambault, A., 2003.

http://www2.cstudies.ubc.ca/~belfer/Papers/202-1335.pdf.

Williams, D., 2000. Evaluation of LOs and instruction

using LOs. In D. A. Wiley, The instructional use of

LOs http://reusability.org/read/chapters/williams.doc

ICEIS 2005 - ARTIFICIAL INTELLIGENCE AND DECISION SUPPORT SYSTEMS

346