ASSESSMENT MODEL FOR IMPROVING EDUCATIONAL

CURRICULUM MATERIALS BASED ON THE DANP

TECHNIQUE WITH GREY RELATIONAL ANALYSIS

Cheng-Hsiung Chen

Department of Information Management, Kainan University, 1, Kainan Rd., Luchu, Taoyuan 338, Taiwan

Gwo-Hshiung Tzeng

Kainan University, 1, Kainan Rd., Luchu, Taoyuan 338, Taiwan

Keywords: Teaching curriculum materials, MCDM (multiple criteria decision making), DANP (DEMATEL-based

ANP), GRA (grey relational analysis).

Abstract: The core objective of the integrated curriculum of compulsory education is to “enable students to

demonstrate their network-talents instead of just scoring high on independent exams.” The key to

determining education reform strategies in the e-era is to establish network-competence indicators for

educational behavior in primary schools. We propose a MCDM means for evaluating, comparing and

improving the effectiveness of network-competence indicators in various publications that are used for

teaching at the primary school level. The Mandarin Chinese teaching curriculum based on this system is

provided to verify the effectiveness of our method, which may extend to other subject areas.

1 INTRODUCTION

Most countries have made the cultivation of human

talent a priority in the twenty-first century. As other

advanced countries propose education reform,

Taiwan also views education as the bedrock of

national development. Taiwan has implemented

various education reforms, such as preschool

education reform, curriculum reforms for grades 1-9,

the restructuring of secondary education, the

enhancement of higher education, and projects to

promote lifelong learning. This study proposes a set

of techniques and evaluation methods to improve,

reconfigure and select the most appropriate Aspiring

Intelligent Grey Relational Assessment System

(AIGRAS) for improving the teaching materials in

our education system.

Four British educational reform projects have

been implemented since 1990 (DfE, 1991). The

report of the Mayer Committee advised the

Australian Education Council and the Ministers for

Vocational Education, Employment and Training on

employment-related key competencies for post-

compulsory education and training (Mayer, 1992).

The Education Commission of Hong Kong proposed

“Learning is the key to one’s future, and education is

the gateway to our society’s tomorrow” (EC, 2000).

Regardless of the style of educational reform,

key competencies are generally the major concern

for national reforms at the beginning of the twenty-

first century.

The core objective of the Nine-Year Integrated

Curriculum of compulsory education in Taiwan is to

“enable students to demonstrate their network-

talents instead of just scoring high on independent

exams.” The key to determining education reform is

to establish network-competence indicators of

targeted educational standards in primary school and

junior high school (MOE, 2002).

Therefore, in this study, we propose a new

MCDM (multiple criteria decision making) method

for evaluating, comparing and improving the

effectiveness of competence indicators in the various

publications used for curriculum materials in

primary schools. The DANP (DEMATEL-based

ANP) (decision making trial and evaluation

laboratory, DEMATEL; analytic network process,

ANP) influential weights are based on getting the

total relationship/influence-related matrix by

DEMATEL technique, using an MCDM approach to

solve and address the network-relational problems of

299

Chen C. and Tzeng G..

ASSESSMENT MODEL FOR IMPROVING EDUCATIONAL CURRICULUM MATERIALS BASED ON THE DANP TECHNIQUE WITH GREY

RELATIONAL ANALYSIS.

DOI: 10.5220/0003320602990308

In Proceedings of the 3rd International Conference on Computer Supported Education (CSEDU-2011), pages 299-308

ISBN: 978-989-8425-50-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

dependence and feedback involving various criteria.

Next, a grey relational analysis (GRA) technique

with DANP influential weights is proposed to

determine and implement the best performance

indicator related to each criterion for improving,

reconfiguring and selecting AIGRAS for the

development of curriculum materials. An empirical

study involving three publishers based on this

system design is provided to verify the effectiveness

of the proposed methods. This design may improve

the efficiency and quality of Mandarin Chinese

teaching curriculum materials; moreover, our work

may also apply to other subject areas.

The remainder of this paper is organized as

follows. In Section 2, AIGRAS for teaching

curriculum materials with MCDM are introduced. In

Section 3, a MCDM method based on the DANP

method is proposed. In Section 4, an empirical study

involving AIGRAS for Mandarin Chinese teaching

materials is presented to demonstrate our proposed

method, and we discuss the results. Finally, in

Section 5, we offer concluding remarks.

2 AIGRAS FOR TEACHING

CURRICULUM MATERIALS

WITH MCDM

In recent decades, competency-based education has

become a major trend, influencing the educational

reform strategies of most governments worldwide.

In the following subsection, we review the related

literatures describing core competencies (CCs) and

the intertwined effects of an assessment system for

teaching materials as a foundation for the

development of a theoretical framework.

2.1 Educational Reform in Taiwan

(MOE, 2002, 2008)

Taiwan must engage in educational reform to meet

the needs of the twenty-first century, respond to

global education reform trends, foster national

competitiveness and boost the overall quality of our

citizens’ lives.

The Ministry of Education (MOE) of Taiwan has

initiated curricular and instructional reforms in

primary school and junior high school education.

These reforms are based on the Action Plan for

Educational Reform approved by the Executive

Yuan of Taiwan. Because the curriculum represents

not only the core of schooling but also the

foundation on which teachers plan learning

activities, the MOE places the greatest emphasis on

the development and implementation of curriculum

reforms for grades 1-9. These timely reforms are

necessary to meet: (1) national development needs

and (2) public expectations with respect to the next

generation.

A major goal of education is to nourish each

student’s mind and character. Every legitimate

government hopes that its school system will

produce outstanding citizens with both a sense of

patriotism and the ability to adopt a global

perspective. In essence, education is a learning

process that helps students explore their potential

and develop their capacity to adapt and improve

their living environments. In this new century, the

following five basic aspects are emphasized and

included in the curricula for grades 1-9:

(1) developing humanitarian attitudes, (2) enhancing

the ability to integrate, (3) cultivating democratic

literacy, (4) fostering both indigenous awareness and

a global perspective, and (5) building a capacity for

lifelong learning.

For both primary schools and junior high

schools, the aim of national education is to teach

students basic networking knowledge and to develop

the capacity for lifelong learning. To cultivate able

citizens, we hope to engender mental and physical

health, vigour and optimism, gregariousness and

helpfulness, intellectual curiosity, reflection,

tolerance, creativity, a positive attitude and a global

perspective. To accomplish this, the curriculum

design of primary school and junior high school

education should focus on the needs and experiences

of students and on developing CCs relevant to

modern citizens. Such CCs are referred to as key

competencies and, as defined by the Mayer

Committee, should: (1) collect, analyze and organize

information; (2) communicate ideas and

information; (3) plan and organize activities;

(4) cooperate with others and help sustain the

group's ability to work; (5) use mathematical

concepts and technologies; (6) solve problems; and

(7) use technology (Mayer, 1992).

In Taiwan, the CCs applicable to curriculum

reforms in grades 1-9 (MOE, 2002) can be

categorized as follows: (1) self-understanding and

exploration of potential; (2) appreciation,

representation, and creativity; (3) career planning

and lifelong learning; (4) expression,

communication, and sharing; (5) respect, care and

teamwork; (6) cultural learning and international

understanding; (7) planning, organizing, and putting

plans into practice; (8) use of technology and

information; (9) active exploration and study; and

(10) independent thinking and problem solving.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

300

With reference to curricular principles, CCs can

be organized into four basic categories: (A) physical,

mental and spiritual mold (1-3); (B) interpersonal

and social relations (4-7); (C) the use of life science

and technology (8); and (D) logical thinking and

reasoning (9-10).

To foster CCs in citizens, the curricula for

primary school and junior high school education

should emphasize three dimensions, including

individual development, community and culture, and

the natural environment. Thus, curricula in grades

1-9 encompasses include seven major learning areas:

(1) Language Arts, (2) Health and Physical

Education, (3) Social Studies, (4) Arts and

Humanities, (5) Science and Technology,

(6) Mathematics, and (7) Integrative Activities.

2.2 AIGRAS with the MCDM Method

“Decision-making is as old as man.” The MCDM

method may be applied for computer-aided learning

(Quaddus, 1997). Most research has concentrated on

evaluating the quality of web-based learning by the

MCDM method (Hwang et al., 2004; Shee & Wang,

2008; Lin, 2010). A MCDM method based on the

DEMATEL technique for evaluating a private

university of science and technology in Taiwan has

been proposed by Tseng (2010).

A MCDM method based on the DEMATEL

technique for assessing Mandarin Chinese teaching

curriculum materials was first proposed by the

authors. MCDM frameworks exist for teaching

curricula so that we can judge the quality of the

teaching materials, but these frameworks are vague,

even if clear CCs are used as the basis for the

criteria. In this paper, we propose an AIGRAS

technique in which the grey relational grade is used

to rank the indices between the performance ratings

of various curriculum materials.

3 A MCDM METHOD BASED

ON THE DANP TECHNIQUE

The structure of the MCDM problem will be derived

using the DEMATEL technique. The priorities of

each determinant are based on the structure derived

by using ANP. The GRA technique will be

leveraged to calculate the degree of alternatives to

be close the aspiring levels. Finally, the assessment

system for obtaining the best teaching curriculum

materials will be derived. In summary, this

evaluation framework consists of four main phases

(see Figure 1).

Figure 1: An analytical framework for the aspiring

assessment systems of teaching materials.

3.1 DANP Technique

The DEMATEL technique was developed by the

Battelle Geneva Institute to analyze complex “world

problems” dealing mainly with interactive man-

model techniques. A second goal was to evaluate

qualitative and factor-linked aspects of societal

problems (Gabus & Fontela, 1972). The

applicability of the method is broad, with

applications ranging from industrial planning and

decision-making to urban planning and design,

regional environmental assessment, the analysis of

global problems, and so forth. This technique has

also been successfully applied in many situations

and contexts, such as creating marketing strategies,

control systems and safety solutions and developing

the competencies of global managers and group

decision-making (Chen et al., 2010; Chiu

et al., 2006; Lee et al., 2009; Li & Tzeng, 2009;

Lin & Wu, 2008; Ou Yang et al., 2008; Wu &

Lee, 2007). Furthermore, a hybrid model combining

the two methods has been widely used in such fields

as e-learning evaluation (Tzeng et al., 2007), airline

safety measurement (Liou et al., 2007), and

innovation policy portfolios for Taiwan's SIP Mall

(Huang et al., 2007).

In this paper, we use DEMATEL not only to

detect complex relationships and build a network

relation map (NRM) of the criteria but also to

calculate the inter-relational influence levels of each

element. We developed/adopted these influence

level values as the basis of the normalization

supermatrix for determining ANP influential weights

of each criterion. To apply DEMATEL, we refined

the definitions based on the above references and

produced new essential definitions, as indicated

below. We based the DEMATEL technique on

graph theory so that we could divide multiple

criteria into cause and effect groups. Directed

influence graphs (also called digraphs) are more

useful than directionless graphs. A digraph typically

represents a communication network-relation or a

domination relationship between individuals.

Suppose that a system contains a set of elements

{

}

12 n

Sss s= 

,, and that particular pair-wise

relationships are used for modeling with respect to a

mathematical relationship (MR). Next, consider the

ASSESSMENT MODEL FOR IMPROVING EDUCATIONAL CURRICULUM MATERIALS BASED ON THE DANP

TECHNIQUE WITH GREY RELATIONAL ANALYSIS

301

relationship MR as a direct-relation matrix that is

indexed equally in both dimensions using elements

from set S. Then, extract the case for which the

number appears in the cell

(

)

i,j

. If the entry is a

positive integral, it means the ordered pair

(

)

in the relationship MR; and its relationship is such

that

has an effect on

. The digraph portrays a

contextual relationship between the elements of the

system in which a numeral represents the strength of

influence (Figure 2). The number between factors is

the degree of influence. For example, an arrow from

represents the fact that

influences

and

that its degree of influence is 2. The DEMATEL

method can convert the relationship between the

causes and effects of criteria into an intelligible

network-structural model of the system (Chiu et al.,

2006).

Figure 2: An example of a directed graph.

The pair-wise comparison scale features five

levels, where the scores 0, 1, 2, 3 and 4 represent the

range from “no influence (0)” to “very high

influence (4)” By experts.

11 12 1

21 22 2

nn nn

aa a

⎡⎤

⎢⎥

⎣⎦







The initial direct-relation matrix

A is the

matrix obtained by pair-wise comparisons in terms

of influences and directions between the

determinants in which

a is denoted as the degree to

which the

determinant affects the

j determinant.

The normalized direct-relation matrix

can be

calculated from Equation (1) in which all principal

diagonal elements are equal to zero.

(1)

where

1maxmax ,max

ij ij

in jn

zaa

≤≤ ≤ ≤

⎛⎞

⎜⎟

⎝⎠

∑∑

In this case,

is called the normalized matrix.

Then, the total relationship / influence-related matrix

T can be obtained

()

∞

−

=+ ++ = −TNN N NIN

(2)

where

I stands for the identity matrix,

is a

direct influence matrix and

⎡⎤

⎣⎦

∞

N

stands for an indirect influence matrix.

x≤<,

≤

∑

≤

∑

and at least one column sum

∑

or one row sum

∑

equals 1, but not all;

hence,

lim 0

→∞

⎡

⎤

⎣

⎦

. The element

t of matrix T

denotes the direct and indirect influences of factor

on factor

j .

The row sum

iij

∑

and column sum

jij

∑

of the total-relation matrix

T are denoted through

{

}

,, 1,2,,

tij n

⎡⎤

=∈

⎣⎦

T 

, then

iij

⎡⎤

==⎡⎤

⎢⎥

⎣⎦

⎢⎥

⎣⎦

∑

and

jij

′

⎡

⎤

⎡⎤

⎢

⎥

⎣⎦

⎣

⎦

∑

where the vectors

and

denote

the sums of the rows and columns, respectively.

Suppose that

r denotes the row sum of the

row of matrix

T , and,

c denotes the column sum of

the

j column of matrix T , when ij= , ()

rc+

represents the index indicating the strength of both

the dispatching and receiving influences.

Furthermore,

(- )

rc is the degree of the central role

that factor

plays in the problem. If (- )

rc is

positive, then factor

will primarily exert influence

upon the strength of other factors, and if

(- )

rc is

negative, then factor

will primarily be influenced

by other factors (Huang et al., 2007; Liou et al.,

2007).

Consequently, the ANP method, a multi-criteria

theory of measurement developed by Saaty (1996),

provides a general framework to deal with decision-

making problems without making assumptions about

the independence of higher-level elements from

lower-level elements and about the independence of

the elements within a level, as in a hierarchy. ANP is

different from traditional MCDM methods (Saaty,

2005). For example, AHP (Analytic Hierarchy

Process), TOPSIS, ELECTRE, et al. usually assume

independence between criteria. ANP is a new theory

that extends AHP to address dependence in feedback

and utilizes the supermatrix approach (Saaty, 2003).

ANP is a more reasonable tool for dealing with

complex MCDM problems in the real world. ANP

features two parts. The first consists of a control

hierarchy or network of criteria and sub-criteria that

control all interactions. The second is a network of

influences among the elements and clusters. A

control hierarchy is a hierarchy of criteria and sub-

criteria for which priorities are derived in the usual

way with respect to the goal of the system being

CSEDU 2011 - 3rd International Conference on Computer Supported Education

302

considered. However, we build the hierarchical

structure of the network relation map (NRM) with

dependency and feedback problems in real situations,

as shown in Figure 3.

Source: Saaty 1996

Figure 3: The control hierarchy.

The analysis of priorities in a system can be

considered in terms of a control hierarchy, with

dependence among its bottom-level alternatives

arranged as a network relation, as shown in Figure 3.

Dependence can occur both within and between

components.

Therefore, a hierarchy structure of NRM for

decision-makers (such as Figure 3) can be derived

by the DEMATEL technique. Based on NRM, a

supermatrix W for ANP as clusters

C , 1, ,hm=  can

be obtained, where cluster

C resides in the h

dimension; we assume that

C has

n elements

(determinants), which we denote as

, , ,

hh hn

ee e

The influences of a given set of elements

(determinants) in a component (dimension) on any

element in the decision system are represented by a

ratio scale priority vector derived from pair-wise

comparisons of the relative importance of one

criterion to another criterion, with respect to the

interests or preferences of the decision-makers. This

relative importance value can be determined using a

scale of 1–9 to represent equal importance to

extreme importance (Saaty, 1996). The supermatrix

is as follows:

11 1 2 1 2 1

nnmmn

11 12 1m

21 22 2m

m1 m2 mm

eeee ee

⎡⎤

⎢⎥

⎣⎦

CC C

WW W













 





A typical entry

W in the supermatrix is called a

block of the supermatrix in the following form,

where each column of

W is a principal eigenvector

of the influence of the elements (determinants) in the

component of the network on an element

(determinants) in the

j component. Some of the

entries may be zero, corresponding to those elements

(determinants) that have no influence.

11 12 1

22 22 2

nn nn

ii ij

ij ij ij

ww w

⎡

⎤

⎢

⎥

⎢

⎥

⎢

⎥

⎢

⎥

⎢

⎥

⎢

⎥

⎣

⎦







After forming the supermatrix, the weighted

supermatrix is derived by precisely transforming the

sum of all columns to unity for normalization. This

step is similar to the concept of the Markov chain in

terms of ensuring that the sum of the probabilities of

all states equals 1. Next, the weighted supermatrix is

raised to limiting powers, such as

lim

→∞

, to obtain

the constant values of global priority vectors, or so-

called weights (Huang et al., 2005).

3.2 Grey Relation for Evaluation

Since Deng (1982) proposed the grey theory, related

models have been developed and applied to MCDM

problems. Similar to fuzzy set theory, the grey

theory is a practical mathematical approach that can

be used to deal with systems analysis characterized

by inadequate information. Fields covered by the

grey theory include systems analysis, data

processing, modeling, prediction, decision-making,

and control engineering (Deng, 1985, 1988, 1989;

Tzeng et al., 2002; Tzeng & Tasur, 1994). In this

section, we briefly review the calculation process for

the grey relation model. This research modifies the

definitions used by Chiou and Tzeng (2001). GRA is

used to determine the relationship between two

sequences of stochastic data in a grey system. The

procedure bears some similarity to pattern

recognition technology. One sequence of data is

called the “reference pattern” or “reference

sequence,” and the correlation between the other

sequence and the reference sequence remains to be

identified (Deng, 1986; Tzeng & Tasur, 1994; Wu et

al., 1996). In this study, we use these concepts to

determine how to improve the degree of grey

relations from the performance values to reach the

aspired values for various publishers (called

alternatives) who create textbooks for Taiwanese

school children.

Let the initial relationship matrix

A be a

matrix, where there are

alternatives and

criteria, obtained by surveying the relationships

following normalization, such as

ASSESSMENT MODEL FOR IMPROVING EDUCATIONAL CURRICULUM MATERIALS BASED ON THE DANP

TECHNIQUE WITH GREY RELATIONAL ANALYSIS

303

1111

** * *

Criteria

Alternatives

(1) ( ) ( )

(1)()()

Aspiring value (1) ( ) ( )

kkkk

mmmm

ccc

www

xxxjxn

xx xj xn

⎡⎤

⎢⎥

⎣⎦











Therefore, coefficients of grey relation for the

aspiring values are

((), ())

jx j

∗

min min | ( ) ( ) | max max | ( ) ( ) |

| ( ) ( )| maxmax| ( ) ( )|

xj xj xj xj

∗∗

−+ −

(3)

Then, the grade (degree) of the grey relation is

obtained so that larger is better:

(,) ((),())

kj k

xx w x jx j

γγ

∗∗

∑

(4)

where the weight

w can be obtained by DANP.

4 AN EMPIRICAL STUDY OF

AIGRAS FOR MANDARIN

CHINESE TEACHING

MATERIALS

In this section, an example modified from a real case

will be presented to demonstrate the effectiveness of

the proposed MCDM framework with the DANP

technique including the grey relational assessment.

One empirical example focuses on the experiences

of three leading textbook publishers.

In this case study, we look at Mandarin Chinese

curriculum materials (six textbooks) for primary

school children in grade 1.

4.1 Background (MOE, 2002, 2008)

In the twenty-first century, major changes have

taken place in social, political, economic and

cultural arenas. These changes are both national and

global. Given the drastic changes brought about by

the e-era, most countries have become aware of the

importance of education and culture. Educational

reform in these countries has been carried out to

foster personal potential, to modernize, and to

promote social progress.

After six decades of post-war development,

Taiwan (also called Formosa) has transitioned from

a traditional agricultural society into a modern

industrial society. Political, economic, and cultural

arenas are facing modernization, industrialization

and the technological influences of structural

adjustment and reconstruction in the e-era. Among

the major arenas of reform, the impact of education

reform is one of the most far-reaching and has

extensive implications. It affects national pride,

impacts social consciousness, establishes a new

culture and develops the nation’s competitiveness in

the new century.

With the spread of education and enhancements

in quality of life, Taiwan is becoming an educational

community. However, many problems have emerged

in the development of education over the years, and

delays in solving these problems have made them

even more complex.

In light of this, the Council on Education Reform

of Taiwan was established in September 1994. The

Commission’s report on educational reform and the

development of educational research was presented

in December 1996.

In Taiwan, certain issues were addressed to

enable reform. The influence of educational reform

at both social and personal levels is significant. It is

important to ponder the impact of social change and

carefully consider how the value of the benchmarks

of socio-cultural development may be determined.

This is especially true in the context of educational

reform.

The Council has proposed a comprehensive

proposal for education reform. Given the need to

provide proper education for every student, it is clear

that schools have not paid adequate attention to

disadvantaged students. This has been mainly due to

inappropriate compulsory education of Taiwan,

where an excessively rigid system and curriculum is

coupled with a lack of long-term investment in

resources. As a result of weak educational practices,

many disadvantaged students fail to build a solid

foundation for learning; they are then exposed to

large class sizes and a general lack of timely and

adequate care. Consequently, their performance falls

even further behind that of other students, causing

them to feel insecure about school.

At the convening of the Education Reform

Committee, the MOE of Taiwan proposed

curriculum reforms for grades 1-9 (compulsory

education) (MOE, 2002, 2008). A more detailed

description is in Subsection 2.1.

The Mandarin Chinese curriculum is one of the

curricula included in Language Arts. It is divided

into three stages: grades 1-3, grades 4-6, and grades

7-9.

Based on curricular goals and CCs for grades 1-

9, the goals of the Mandarin Chinese curriculum

included listening, speaking, reading and writing of

CSEDU 2011 - 3rd International Conference on Computer Supported Education

304

languages, as well as developing basic

communication competencies.

Based on the requirements for children's

intellectual development, the numbers of

competence indicators of the Mandarin Chinese

curriculum for each stage are presented below (see

Table 1).

Table 1: Competence indicators for the Mandarin Chinese

curriculum at each stage.

CCs

Stage

Total

1 2 3

: Physical, mental and spiritual mold

: Self-understanding and exploration of potential 24 19 18 61

: Appreciation, representation, and creativity 20 17 17 54

: Career planning and lifelong learning 9 13 10 32

: Interpersonal and social relations

: Expression, communication, and sharing 8 9 12 29

: Respect, care and teamwork 9 9 13 31

: Cultural learning and international understanding 6 8 6 20

: Planning, organizing and putting plans into

practice

7 7 6 20

: The use of life science and technology

: Using technology and information 4 11 8 23

: Logical thinking and reasoning

: Active exploration and study 8 7 9 24

: Independent thinking and problem solving 9 8 7 24

Total 104 108 106 318

Data source: MOE, 2002

Based on our chosen analytical framework, CCs

were first selected as determinants. The structure of

the assessment systems for the Mandarin Chinese

teaching materials was then established using

DANP. Then, the influential weights of each

determinant for the decision structure was identified.

The determinants are equivalent to CCs in this

research. The criteria were confirmed as competence

indicators and as determinants for improving

Mandarin Chinese teaching materials. Meanwhile,

the relationships between the determinants, the

DANP derivations of the weight of each

determinant, and the GRA of each determinant were

similarly derived for our case study.

4.2 Structuring NRM and Calculating

the Influential Weights of

Determinants using DANP

The relationships between determinants involving

assessment systems for Mandarin Chinese teaching

materials were surveyed based on the opinions of

teachers in Taiwan who write Mandarin Chinese

textbooks (1/3), have taught Mandarin Chinese in

primary school (1/3), or have taught other subjects in

primary school (1/3). All of the surveyed teachers

were familiar with the assessment protocols for

Mandarin Chinese in schools.

Our proposed assessment system for Mandarin

Chinese will assist publishers in creating better

textbooks and will help to determine which

assessment strategies can best achieve the aspiring

levels of quality of textbooks. Detailed procedures

and results are given below.

Table 2: Rating the CCs’ relationships matrix A for grades

1-3: Mandarin Chinese curricula.

0.000 3.500 3.375 3.625 3.250 2.250 2.750 2.375 2.875 3.000

3.250 0.000 2.875 3.000 2.625 2.875 2.750 2.250 2.500 2.375

3.750 2.375 0.000 2.875 2.625 2.500 3.000 2.750 3.000 3.125

3.750 3.250 3.000 0.000 3.250 3.125 3.250 2.750 3.125 3.250

2.375 3.000 2.125 3.500 0.000 3.500 3.375 2.375 2.750 3.000

2.625 3.000 2.500 3.250 3.125 0.000 2.875 2.375 2.625 2.625

2.625 2.625 3.375 3.375 3.375 2.875 0.000 2.750 3.000 3.125

2.375 2.500 2.875 3.125 2.375 2.500 3.000 0.000 3.250 3.250

3.000 3.000 3.375 3.250 3.000 2.875 3.500 3.125 0.000 3.750

3.375 3.375 3.125 3.000 2.875 2.625 3.250 2.875 3.875 0.000

Table 3: Results of the CCs’ total relationships matrix T

for grades 1-3: Mandarin Chinese.

1.1989 1.2839 1.2798 1.3786 1.2721 1.1868 1.3055 1.1287 1.2787 1.3022

1.1962 1.0744 1.1645 1.2523 1.1539 1.1083 1.2002 1.0342 1.1654 1.1802

1.2752 1.2146 1.1384 1.3175 1.2173 1.1574 1.2736 1.1063 1.2450 1.2678

1.3819 1.3446 1.3369 1.3402 1.3398 1.2754 1.3896 1.1999 1.3542 1.3787

1.2329 1.2298 1.2038 1.3322 1.1317 1.1851 1.2818 1.0928 1.2346 1.2607

1.1988 1.1890 1.1737 1.2813 1.1885 1.0379 1.2253 1.0562 1.1897 1.2083

1.2867 1.2638 1.2847 1.3773 1.2810 1.2097 1.2242 1.1445 1.2880 1.3116

1.2067 1.1886 1.1998 1.2927 1.1809 1.1305 1.2441 0.9939 1.2232 1.2419

1.3678 1.3436 1.3544 1.4480 1.3390 1.2745 1.4038 1.2172 1.2641 1.4001

1.3587 1.3351 1.3284 1.4204 1.3162 1.2490 1.3767 1.1927 1.3628 1.2651

The interrelationships between the ten determinants

were deduced using the DANP method in

Subsection 3.1. First, the direct relation matrix

was introduced (see Table 2). After that, the direct

relation matrix

A was normalized based on

Equation (1). The total relationship matrix was then

deduced based on Equation (2) (see Table 3).

Finally, the strength of the influence for each

determinant was deduced (see Table 4 and Figure 4).

Table 4: r

and r

-c

for the CCs’ total relationships for

grade 1-3: Mandarin Chinese.

+ c

- c

: Physical, mental and spiritual mold

24.6645(3) -0.4259(4)

: Self-understanding and exploration of potential 25.3189(5) -0.0886(5)

: Appreciation, representation, and creativity 23.9970(8) -0.9377(10)

: Career planning and lifelong learning 24.6777(6) -0.2515(8)

: Interpersonal and social relations

25.1368(2) -0.1634(3)

: Expression, communication, and sharing 26.7817(1) -0.0992(6)

: Respect, care and teamwork 24.6058(7) -0.2351(7)

: Cultural learning and international understanding 23.5633(9) -0.0659(4)

: Planning, organizing and putting plans into

practice

25.5963(4) -0.2534(9)

: The use of life science and technology

23.0685(4) 0.7357(1)

: Using technology and information 23.0685(10) 0.7357(2)

: Logical thinking and reasoning

26.0200(1) 0.5977(2)

: Active exploration and study 26.0180(3) 0.8069(1)

: Independent thinking and problem solving 26.0219(2) 0.3886(3)

ASSESSMENT MODEL FOR IMPROVING EDUCATIONAL CURRICULUM MATERIALS BASED ON THE DANP

TECHNIQUE WITH GREY RELATIONAL ANALYSIS

305

Figure 4: Causal diagram of the total influential

relationship.

Table 5: Weights of the determinants derived by DANP.

Local Global

: Physical, mental and spiritual mold

0.3015

: Self-understanding and exploration of potential

0.3376 0.1018

: Appreciation, representation, and creativity

0.3312 0.0998

: Career planning and lifelong learning

0.3312 0.0998

: Interpersonal and social relations

0.4054

: Expression, communication, and sharing

0.2656 0.1077

: Respect, care and teamwork

0.2455 0.0995

: Cultural learning and international understanding

0.2335 0.0947

: Planning, organizing and putting plans into

practice

0.2554 0.1035

: The use of life science and technology

0.0895

: Using technology and information

1.0000 0.0895

: Logical thinking and reasoning

0.2037

: Active exploration and study

0.4959 0.1010

: Independent thinking and problem solving

0.5041 0.1027

With an appropriate assessment system as the goal,

pair-wise comparisons of the determinants were

calculated based on the total relationship matrix, as

deduced by DEMATEL. Note the interrelationships

between the goals and the directions of arrows,

relationship matrix serves as a set of inputs for ANP.

By implementing ANP, the limit supermatrix

can

be calculated.

Weights corresponding to each determinant

(Table 5) are derived accordingly and may be used

to calculate both weighted averages and GRA

scores.

4.3 Compromise Rankings Calculated

using GRA

After the determinants’ weights were calculated

using DANP, the GRA technique introduced for

compromise ranking was applied. Weighted

averages of the six Mandarin Chinese textbooks for

grade 1 of primary school, which were edited by

three leading publishers (Table 6), were also

calculated as comparisons. In general, the calcu-

lation results (Table 7) demonstrated that the same

conclusions are apparent at both the global and local

levels: publisher A



publisher C



publisher B.

Table 6: Satisfaction of grade 1 textbooks.

B C

Dimensions Criteria 11

11 12 11 12

6.0000 5.7500 5.5000 5.2500 6.2500 6.2500

6.5000 6.7500 5.5000 7.0000 5.7500 7.2500

5.7500 4.7500 4.2500 4.5000 6.2500 4.7500

6.7500 7.5000 6.0000 6.5000 6.2500 7.2500

6.7500 6.2500 6.2500 6.5000 6.5000 6.5000

5.2500 4.0000 4.2500 4.2500 4.0000 4.0000

5.0000 5.0000 4.5000 4.7500 4.5000 4.2500

3.7500 3.7500 4.0000 4.7500 4.5000 4.5000

5.0000 5.2500 4.2500 4.2500 4.7500 4.5000

5.2500 4.7500 4.7500 4.5000 5.0000 6.0000

Note: a::11: the 1

Semester of grade 1; b: 12: the 2

Semester of grade 1

Table 7: Grey relations versus weighted average results.

Local Global

B C

11 12 11 12

0.3015

0.8003(5) 0.7722(3) 0.7033( 1) 0.7613(2) 0.7998(4)0. 8158(6)

0.3376 0.1019

0.7895(4) 0.7627(3) 0.7377( 2) 0.7143(1) 0.8182(5)0. 8182(5)

0.3312 0.0998

0.8491(3) 0.8824(4) 0.7377( 1) 0.9184(5) 0.7627(2)0. 9574(6)

0.3312 0.0998

0.7627(5) 0.6716(3) 0.6338( 1) 0.6522(2) 0.8182(6)0. 6716(3)

0.4054

0.7946(6) 0.7872(5) 0.7251( 1) 0.7535(3) 0.7363(2)0. 7686(4)

0.2656 0.1077

0.8824(4) 1.0000(6) 0.7895( 1) 0.8491(3) 0.8182(2)0. 9574(5)

0.2455 0.0995

0.8824(6) 0.8182(1) 0.8182( 1) 0.8491(3) 0.8491(3)0. 8491(3)

0.2335 0.0947

0.7143(6) 0.6164(1) 0.6338( 4) 0.6338(4) 0.6164(1)0. 6164(1)

0.2554 0.1035

0.6923(5) 0.6923(5) 0.6522( 2) 0.6716(4) 0.6522(2)0. 6338(1)

0.0895

0.6000(1) 0.6000(1) 0.6164( 3) 0.6716(6) 0.6522(4)0. 6522(4)

1.0000 0.0895

0.6000(1) 0.6000(1) 0.6164( 3) 0.6716(6) 0.6522(4)0. 6522(4)

0.2037

0.7034(5) 0.6928(4) 0.6529( 2) 0.6431(1) 0.6821(3)0. 7214(6)

0.4959 0.1010

0.6923(5) 0.7143(6) 0.6338( 1) 0.6338(1) 0.6716(4)0. 6522(3)

0.5041 0.1027

0.7143(5) 0.6716(2) 0.6716( 2) 0.6522(1) 0.6923(4)0. 7895(6)

ζ=0.5

0.7603(5) 0.7467(4) 0.6941( 1) 0.7260(2) 0.7368(3)0. 7628(6)

Note: c::11: the 1

Semester of grade 1; d: 12: the 2

Semester of grade 1.

4.4 Discussions and Implications

Authoring teaching materials is not an easy task.

There are no straightforward answers to the question

of how teaching materials should be designed to

meet particular criteria and determinants,

considering that textbooks should generally take into

account the relevant curriculum. Very little research

addresses assessment systems for Mandarin Chinese

teaching materials.

In this study, a MCDM framework that

combined the DANP technique and GRA was

proposed to address the aforementioned problems.

We consider the results satisfactory. This MCDM

framework was created to: (a) overcome the issue of

defining the assessment system for teaching

materials, (b) use innovative but traditional MCDM

approaches to resolve the problem of how to define

an assessment system for textbooks, (c) explore the

vague correlations between the determinants of

teaching materials, (d) create target priorities for the

resulting teaching materials, and (e) shape vague

semantic processing issues, such as defining ‘good’

and ‘very good’ assessments.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

306

CCs were selected as determinants for the

assessment system. CCs were then categorized into

four groups and used as criteria for the assessment.

These groups were as follows:

: Physical, mental and spiritual mold

: Self-understanding and exploration of

potential

: Appreciation, representation, and creativity

: Career planning and lifelong learning

: Interpersonal and social relations

: Expression, communication, and sharing

: Respect, care and teamwork

: Cultural learning and international

understanding

: Planning, organizing and putting plans into

practice

: The use of life science and technology

: Using technology and information

: Logical thinking and reasoning

: Active exploration and study

: Independent thinking and problem solving

Through application studies, we found the MCDM

model to be relevant and helpful. DANP establishes

a reasonable assessment structure for dealing with

the influence of various criteria. The influence

relationships pertaining to the results (see Figure 4)

were quite reasonable.

(1) The goal of education is to nurture each

student’s mind and character. First, enhancing

“the use of life science and technology” may

strengthen “logical thinking and reasoning” for

students, which then improves “interpersonal and

social relations” and becomes the ultimate goal;

finally, this gives students “the physical, mental

and spiritual dimensions.”

(2) For logical thinking and reasoning,

“active exploration and study” is better than

“independent thinking and problem solving,”

consistent with the well-known point of view of

knowledge management.

(3) For interpersonal and social relations,

starting from “cultural learning and international

understanding” may shorten the mental distance

for the individual; sharing “expression,

communication” should then naturally lead to

improvement in “respect, care and teamwork,”

and “planning, organizing and putting plans into

practice” should be thoroughly examined.

(4) For physical, mental and spiritual mold,

“self-understanding and exploration of potential”

is the benchmark of “career planning and lifelong

learning” and “appreciation, representation, and

creativity.”

DANP suggests a general framework for dealing

with decisions without making assumptions about

the independence of higher-level elements from

lower-level elements and about the independence of

the elements within a typical hierarchy. Therefore,

by defining the assessment system for teaching

materials as illustrated in the paper, DANP is

apparently a more reasonable tool for analyzing

network structures with feedback.

The GRA technique uses a grade of the grey

relation function

(,)

∗

that represents “closeness

to the aspiring level.” The GRA approach in this

paper simply measures the grey relationship of the

alternatives with the aspiring level.

In Table 7, globally, the six volumes for

(,)

∗

are 0.7603, 0.7467, 0.6941, 0.7260, 0.7368, and

0.7628, and the sequences are 5, 4, 1, 2, 3, and 6,

respectively. This means that Book 11 of Publisher

≺ Book 12 of Publisher B ≺ Book 11 of

Publisher C

≺ Book 12 of Publisher A ≺ Book 11

of Publisher A

≺ Book 12 of Publisher C.

However, if Publisher B were to improve Book

11, for example, the sequences would be (1) the use

of life science and technology (

= 0.6164

), (2)

logical thinking and reasoning (

0.6529

), (3)

physical, mental and spiritual mold (

0.7033

then (4) interpersonal and social relationships

(

0.7251

5 CONCLUDING REMARKS

This paper advances work in the field of teaching

material assessment. In response to current concerns,

we began with Mandarin Chinese. First, a MCDM

framework was proposed to define the determinants

of any set of teaching materials (not exclusively for

Mandarin Chinese). Second, the traditional problem

of the difficulty of defining an assessment system

for teaching materials was resolved using the

MCDM approach. An important conclusion was that

the most important determinants may be the least

influential determinants. We concluded that the

sequence of influence among determinants is as

follows: the use of life science and technology (D

)

 logical thinking and reasoning (D

) 

interpersonal and social relations (D

)  physical,

mental and spiritual mold (D

) (see Figure 4).

However, the weighting sequence of determinants is

as follows: interpersonal and social relations (D

0.405

)

 physical, mental and spiritual mold (D

0.301

)

 logical thinking and reasoning (D

0.2037)

 the use of life science and technology

, 0.0895) (see Table 5). Finally, the challenge of

ASSESSMENT MODEL FOR IMPROVING EDUCATIONAL CURRICULUM MATERIALS BASED ON THE DANP

TECHNIQUE WITH GREY RELATIONAL ANALYSIS

307

selecting “rotten apple(s)” when using the traditional

MCDM approach was also addressed based on our

conceptual advances in achieving the aspiring level

for each of the criteria.

REFERENCES

Chen, Y. C., Lien, H. P., & Tzeng, G. H., 2010. Measures

and evaluation for environment watershed plan using a

novel hybrid MCDM model. Expert Systems with

Applications, 37(2), 926-938.

Chiou, H. K., & Tzeng, G. H., 2001. Fuzzy hierarchical

evaluation with Grey relation model of green

engineering for industry. International Journal of

Fuzzy Systems, 3(3), 466-475.

Chiu, Y. J., Chen, H. C., Tzeng, G. H., & Shyu, J. Z.,

2006. Marketing strategy based on customer behavior

for the LCD-TV. International Journal of

Management and Decision. Making, 7(2/3), 143-165.

Deng, J. L., 1982. Control Problems of Grey Systems.

Systems and Control Letters, 5(2), 288-294.

Deng, J. L., 1985. Fundamental Methods of Grey Systems.

Wuhan, China: Huazhoug University of Science and

Technology.

Deng, J. L., 1986. Grey Forecasting and Decision.

Wuhan: Huazhong University of Science and

Technology Press.

Deng, J. L., 1988. Grey System Book. Windsor: Science

and Technology Information Services.

Deng, J. L., 1989. Introduction of Grey Theory. Journal of

Grey System, 1(1), 1-24.

Department for Education, 1991. Education and training

for the 21st century. London: HMSO.

Education Commission, 2000. Reform Proposal for the

Education System in Hong Kong. Hong Kong:

Education Bureau.

Gabus, A., & Fontela, E., 1972. World Problems, an

Invitation to Further Thought within the Framework of

DEMATEL. Geneva, Switzerland: Battelle Geneva

Research Centre.

Huang, C. Y., Shyu, J. Z., & Tzeng, G. H., 2007. Recon-

figuring the innovation policy portfolios for Taiwan's

SIP Mall industry. Technovation, 27(12), 744-765.

Huang, J. J., Tzeng, G. H., & Ong, C. S., 2005.

Multidimensional data in multidimensional scaling

using the Analytic Network Process. Pattern

Recognition Letters, 26(6), 755-767.

Hwang, G. J., Huang, T. C. K., & Tseng, J. C. R., 2004. A

group-decision approach for evaluating educational

web sites. Computers & Education, 42(1), 65-86.

Li, C. W., & Tzeng, G. H., 2009. Identification of

Interrelationship of Key Customers' Needs Based on

Structural Model for Services/Capabilities Provided by

a Semiconductor -Intellectual Property Mall. Applied

Mathematics and Computation, 215(6), 2001-2010.

Lin, C. J., & Wu, W. W., 2008. A causal analytical

method for group decision-making under fuzzy

environment. Expert Systems with Applications, 34(1),

205-213.

Lin, H. F., 2010. An application of fuzzy AHP for

evaluating course website quality. Computers &

Education, 54(4), 877-888.

Liou, J. J. H., Tzeng, G. H., & Chang, H. C., 2007. Airline

safety measurement using a hybrid model. Journal of

Air Transport Management, 13(4), 243-249.

Mayer, E., 1992. Putting general education to work: The

key competencies report. Canberra: Australian

Government Publishing Service.

Ministry of Education, 2002. General Guidelines of Grade

1-9 Curriculum of Primary and Junior High School

Education. Taipei: Ministry of Education.

Ministry of Education, 2008. Education in Taiwan 2008,

Taipei: Ministry of Education.

Ou Yang, Y. P., Shieh, H. M., Leu, J. D., & Tzeng, G. H.,

2008. A novel hybrid MCDM model combined with

DEMATEL and ANP with applications. International

Journal of Operations Research, 5(3), 160-168.

Quaddus, M. A., 1997. Computer aided learning of

multiple criteria decision making: An integrated

system and its effectiveness. Computers & Education,

28(2), 103-111.

Saaty, R. W., 2003. The Analytic Hierarchy Process

(AHP) for Decision Making and the Analytic Network

Process (ANP) for Decision Making with Dependence

and Feedback. Pittsburgh, PA: Creative Decisions

Foundation.

Saaty, T. L., 1996. Decision Making with Dependence and

Feedback: The Analytic Network Process. Pittsburgh,

PA: RWS Publication.

Saaty, T. L., 2005. Theory and Applications of the

Analytic Network Process. Pittsburg, PA: RWS

Publications.

Shee, D. Y., & Wang, Y. S., 2008. Multi-criteria

evaluation of the web-based e-learning system: A

methodology based on learner satisfaction and its

applications. Computers & Education, 50(3), 894-905.

Tseng, M. L., 2010. Implementation and performance

evaluation using the fuzzy network balanced

scorecard. Computers & Education, 55(1), 188-201.

Tzeng, G. H., Chang, J. R., Lin, J. D., & Hung, C. T.,

2002. Non-Additive Grey Relation Model for the

Evaluation of Flexible Pavement. International

Journal of Fuzzy Systems, 4(2), 715-724.

Tzeng, G. H., Chen, W. H., Yu, R., & Shih, M. L., 2010.

Fuzzy Decision Maps: a generalization of the

DEMATEL methods. Soft Computing, 14(11), 1141-

1150.

Tzeng, G. H., Chiang, C. H., & Li, C. W., 2007.

Evaluating intertwined effects in e-learning programs:

A novel hybrid MCDM model based on factor

analysis and DEMATEL. Expert Systems with

Applications, 32(4), 1028-1044.

Tzeng, G. H., & Tasur, S. H., 1994. The Multiple Criteria

Evaluation of Grey Relation Model. Journal of Grey

Systems, 6(2), 87-108.

Wu, H. S., Deng, J. L., & Wen, K. L., 1996. Introduction

of Grey Analysis. Taiwan: Gau-Li Publication Inc.

Wu, W. W., & Lee, Y. T., 2007. Developing global

managers' competencies using the fuzzy DEMATEL

method. Expert Systems with Applications, 32(2), 499-

507.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

308