DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL

EVALUATION SYSTEM USING COMMONKADS

Javier Andrade, Juan Ares, Rafael García, Santiago Rodríguez and Sonia Suárez

Software Engineering Laboratory, University of A Coruña, Campus de Elviña s/n,15071, A Coruña, Spain

Keywords: CommonKADS, Clips, Knowledge-Based System, Knowledge Management, KPQM.

Abstract: Several knowledge management maturity models have been proposed in the last years. These models are

used to evaluate the quality of knowledge management practices in the organizations. One of these models

is the Knowledge Process Quality Model, which has five maturity levels. The acquisition of a high maturity

level is usually expensive due to the evaluations and improvement processes that are often required for a

positive final decision. With the aim of minimizing these costs, this paper proposes a Knowledge-Based

System that tries to check if the company currently stands in compliance with a given KPQM maturity level.

The actual evaluation process starts only if the system output is positive. This approach implies an important

cost reduction by avoiding negative evaluations. The design of the system is based on the CommonKADS

methodology, and its implementation was carried out with the Clips tool.

1 INTRODUCTION

According to Kuriakose et al. (2011), Knowledge

Management (KM) is a discipline that tries to create

wealth and value by providing the right knowledge

at the right place and at the right time. The effective

use of knowledge by several organizational entities

results in improved skills and competencies for

decision making, performance improvement and

also innovation. Thus, the organizations had devoted

numerous efforts for retaining and institutionalizing

the knowledge they possess (Davenport and Prusak,

2000). In this way, there are several KM initiatives

in the literature (Ares et al., 2008) that try to

establish guides for effectively implementing and

developing the KM in the organizations.

The following obvious step involves the set up of

mechanisms for assessing how well these KM

initiatives are deployed in the organizations. This

assessment focuses on the quality of the organization

processes, as well as in the mechanisms used for

achieving and keeping these processes. The later

mechanisms should also be used as a guide for

continuous improvement.

As a result, there is a growing number of the so-

called Knowledge Management Maturity Models

(KMMM). Examples of these initiatives are: the

KMf and KM

models (Gallagher and Hazlett,

2000), the KPMG Knowledge Journey (KPMG,

2000), the model proposed by Gabor Klimko (2001),

the Knowledge Management Capability Assessment

(KMCA) (Kulkarni and Freeze, 2004; 2005), the

Knowledge Process Quality Model (KPQM)

(Paulzen and Perc, 2002) and the model proposed by

Infosys (Kochikar, 2000).

The compliance with a high KM maturity level

increases the prestige and competitiveness of the

organizations. However, the evaluation process is

often quite expensive. An auditor has to determine

the compliance with a given KM maturity level. In

the case of non compliance, the defects detected

have to be solved and a new audit is required to the

evaluation of the organization in the desired KM

maturity level.

This paper proposes a Knowledge-Based System

(KBS) for the evaluation of an organisation in a KM

maturity level. This KBS will return a positive or

negative compliance report with regards to the given

KM maturity level. The services of the auditor are

required only when the result of the KBS is positive.

There is a high matching probability between

this report and that of the auditor, but sometimes this

matching may not happen, since the auditor may

consider certain aspects that are not considered by

the KBS. These aspects are also useful, since the

inclusion of new knowledge will improve the KBS.

The KPQM model, proposed by Paulzen and

Perc (2002), was selected because its structure and

459

Andrade J., Ares J., García R., Rodríguez S. and Suárez S..

DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL EVALUATION SYSTEM USING COMMONKADS.

DOI: 10.5220/0003697504590464

In Proceedings of the 4th International Conference on Agents and Artiﬁcial Intelligence (ICAART-2012), pages 459-464

ISBN: 978-989-8425-95-9

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

processes are well detailed. This model is based on

the ideas of two software development maturity

models: the Capability Maturity Model (CMM)

(McCollum, 2006) (Persse, 2001) (SEI, 2011) and

the Software Process Improvement and Capability

dEtermination (SPICE) (SPICE, 2011).

The KPQM model description and the design of

the proposed KBS are detailed in Section 2; Section

3 shortly describes the KBS implementation, and

Section 4 sets out the conclusions.

2 DESIGN OF THE PROPOSED

SYSTEM

2.1 KPQM Description

Starting from the CMM and SPICE models, Paulzen

and Perc (2002) developed a model that allows the

assessment of KM processes in an organization. The

model also provides a summary of the steps to be

followed in order to reach some improvements. The

model includes five maturity levels (see Table 1).

Table 1: Maturity levels.

Maturity Level Description

1. Initial

The quality of the knowledge

processes is not defined and it

randomly changes. This level

could be described as chaotic.

2. Aware

The need of managing

knowledge processes starts to be

borne in mind. The first

structures for guarantying a better

quality of processes are

implemented.

3. Established

A systematic structure and a

definition for knowledge

processes are established.

4. Quantitatively

Manager

The systematic management of

processes is emphasized by

means of performing output

measurements for their planning

and guidance.

5. Optimizing

There are structures for

continuous improvement.

Each maturity level defines a series of issues

with which the organization must comply. That

structure, as Table 2 shows, rests on the following

dimensions: Organization, People and Technology.

The KBS design has to reflect that structure in

order to assess the compliance with all the maturity

levels defined in the KPQM.

Table 2: Assessment structure.

Level Dimension Issue

Organization

People

Technology

Organization

Defined and documented

processes.

Process-native basic abilities

and structures.

People

Structures for the employees

to consider KM processes.

Structures for the direction

to consider KM processes.

Technology

It includes part of the

technological support for

KM methods.

Organization

Process standardization.

Structured knowledge.

People

There is an incentive system

for employees in order to

boost the use of knowledge.

There is an incentive system

for executives in order to

boost the use of knowledge.

Technology

There is a systematic

technological support for the

processes.

Organization

The processes are

quantitatively assessed.

The management decisions

are quantitatively assessed.

People

The incentive system for

employees is quantitatively

assessed.

The incentive system for

executives is quantitatively

assessed.

Technology

The impact of the

technological support is

quantitatively assessed.

Organization

There are structures for

improving the processes.

There are structures for

improving the management.

People

There are structures that

promote the continuous

improvement in the use of

knowledge.

There are structures that

guarantee that the

management might keep

involved in the KM tasks.

Technology

The process-supporting

technologies are optimized.

Pilot projects are carried out.

The quality of the KBS design depends on the

programming skills of the knowledge engineers and

also on their abilities to devise, remember, and

ICAART 2012 - International Conference on Agents and Artificial Intelligence

460

dynamically update a design specification. This is a

difficult task for all but the smallest KBSs.

Difficulties like these can be alleviated by

producing textual representations or diagrams of

expert knowledge and design specifications. The

best known approach for producing such documents

is the CommonKADS methodology

(CommonKADS, 2011) (Schreiber, 2000)

(Kingston, 1998) (Valente, 1998). CommonKADS is

the current European de facto standard for

knowledge analysis and knowledge-intensive

systems development. This methodology has been

wholly or partially incorporated to existing methods

by many major companies in Europe, US and Japan

(CommonKADS, 2011). CommonKADS was used

in this work for elaborating a list of potential model

components for the KBS, selecting the adequate task

template, constructing the initial domain scheme,

and completing the specification of the knowledge

model. The following sections describe how these

activities were carried out.

2.2 List of Potential Model

Components

The task of the proposed KBS belongs to a highly

specialized field (a concrete and classified theme

within Quality Management). Mainly due to this

reason there is information available on how to carry

out audits (SEI, 2011). Consequently, the knowledge

of the domain can be considered as formal.

On the one hand, there is evidence of the

existence of a commonly accepted structure in the

KPQM model—shown in Figure 1—that represents

an initial candidate for the domain model. This

structure reflects the existence of five maturity levels

and three common dimensions. Each dimension is

related to a set of issues in each maturity level.

Also, a maturity level requires a certain level of

compliance with each dimension, and each

dimension contains a series of issues presented by

maturity-shaped questions. We have been able to

define these questions thanks to the KPQM literature

and the information available on how to carry out

audits (c.f., e.g., SEI, 2011).

Figure 1: Initial relationships structure.

On the other hand, it is essential to record the

performed audits and their results (e.g. in a file or

database). Therefore, before resolving the question

about a KM maturity level n, the KBS has to check

if the organization satisfies the KM maturity level n-

1. This practice follows the Quality Management

philosophy, which stands on not skipping maturity

levels. In that way, level 2 should be previously

achieved in order to reach level 3.

2.3 Selection of the Task Template

The aim of the proposed KBS is to fill a form with

organizational information and analyze it in order to

determine if the organization currently stands in

compliance with the selected KPQM maturity level;

that is, if an actual evaluation is possible.

In this context, and from the point of view of the

task, this is an activity that fits into the category of

assessment. These activities are provided with

various templates, from which we have selected the

one mentioned in Schreiber et al. (2000).

The main motive for this choice is that the

associated inferential structure matches the purpose

of the KBS. A good technique to establish this

adequacy is building an annotated inferential

structure in which the roles are annotated with

specific elements of the domain. This inferential

structure is shown in Figure 2.

Figure 2: Annotated inferential structure.

2.4 Construction of the Initial Domain

Scheme

The result of this task is a set of domain-specific

conceptualizations — shown in Figure 3 — and a set

of method-specific conceptualizations — shown in

Figure 4.

Two main concepts were detected in the domain:

Form and Section. The information about the last

KPQM level reached by the organization was also

required and it is represented by the concept Record.

DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL EVALUATION SYSTEM USING COMMONKADS

461

Figure 3: Domain-specific conceptualizations.

Figure 4: Method-specific conceptualizations.

The Form and Record concepts constitute the

initial reasoning case. A Form refers to a specific

KPQM level and consists of three sections each of

them representing a dimension (Organization,

People, and Technology). There is an aggregation

relationship between the concepts Form and Section.

The concept Form has an attribute associated-level

that indicates the desired KPQM level. The concept

Section has four attributes: name, total-questions,

positives, and category. The first refers to the name

of the dimension—e.g. People—, the second

indicates the total number of issue-related questions

in the dimension, the third represents the total

number of positively answered questions, and the

last attribute refers to the level of compliance with

the section. This level of compliance is obtained by

means of the total-questions and positives attributes

as follows:

- If the positive answers represent less than 25%

of the total, the level of compliance is none: the

organization does not comply with the

dimension represented by the section.

- If the positive answers represent between 25-

50% of the total, the level of compliance is low.

- If the positive answers represent between 50-

75% of the total, the level of compliance is

medium.

- If the positive answers represent between 75-

100% of the total, the level of compliance is

high.

Once it is determined how the domain concepts

will be used, there should be settled the criteria to be

applied to the data in order to determine the

compliance with a given KPQM level. In this case,

two different criteria are considered, each with a

attribute truth-value representing whether or not the

criterion is fulfilled:

- Last-level: Was the organization successfully

audited in the level n-1?

- Concrete-level: Does the organization meet the

requirements of the level n? The organization

must meet the issues in the level dimensions at

certain rates or levels (many possibilities are

accepted).

Finally, it should be highlighted that the system

only offers a positive answer if all the criteria have

the true value.

2.5 Complete Specification of the

Knowledge Model

As explained before, the activity to be modelled is

an example of the task type assessment. Also, the

selected template shows an adequate inferential

structure for the purpose of this KBS, in which the

inferences present sufficient detail.

The task that must be carried out is decomposed

into two subtasks, which means that the task method

structures the reasoning process in two steps:

- Abstraction: the purpose of this step is to achieve

the compliance level for each section. As

explained above, this level can be none, low,

medium or high. For the KBS to reason as an

expert does, the meaning of the number of the

positive answers should be known. That is, the

reasoning of an expert auditor may be: The

organization complies with all the dimensions at

a medium level, but the People dimension must

have a high level of compliance in this KPQM

maturity level and I therefore consider that

improvements must be made in that dimension.

- Matching: the abstractions are matched in order

to take the final decision on whether or not the

organization complies with the established

criteria.

Figure 5 shows the template that was chosen for

the modelling.

On the other hand, the knowledge scheme that

was finally obtained is shown in Figure 6. It can be

observed that the final domain scheme incorporates

three rule types:

- Case-abstraction: abstraction rules required for

reaching the section compliance level by using

the attributes total-questions and positives.

ICAART 2012 - International Conference on Agents and Artificial Intelligence

462

assess case

assess through

abstract & match

abstract case match case

abstract

method

match

method

specify select evaluate match

task

task method

task

task method

infe rence

assess case

assess through

abstract & match

abstract case match case

abstract

method

match

method

specify select evaluate match

task

task method

task

task method

infe rence

Figure 5: Decomposition of the task.

- Form-requirement: rules offering truth values

to the criteria Last-level and Concrete-level.

The first one indicates the compliance with the

previous KPQM level and the second one

indicates the acceptable compliance levels for

the desired KPQM level.

- Level-decision-rule: The decision is represented

by a concept Level-decision with an attribute

that indicates whether or not the organization

has real possibilities of successfully

overcoming an audit for the desired KPQM

level. Also, this rule expresses the relationship

between the different criteria and the final

decision taken by the KBS.

form-requirement

level-decision-rule

case-abstraction

Case

indicates

has abstraction

KPQMLevel Criterion

implies

level-decision

form-requirement

level-decision-rule

case-abstraction

Case

indicates

has abstraction

KPQMLevel Criterion

implies

level-decision

Figure 6: Final knowledge scheme.

3 IMPLEMENTATION OF THE

PROPOSED SYSTEM

The system was implemented according to the

CommonKADS design and by means of the Clips

tool (Clips, 2011). In order to provide the

application with modularity and to simplify the

development and depuration processes, the

following knowledge bases were defined:

- General: This knowledge base contains all the

definitions of classes, objects, and properties. It

contains the operative knowledge of the system.

See for example, the concept Section definition:

(defclass SECTION (is-a USER)

(role concrete)

(slot name (type STRING))

(slot total-questions (type INTEGER))

(slot positives (type INTEGER))

(slot category (allowed-values none low medium

high) (default none))

and the object Org (KPQM dimension

Organization) definition:

(definstances SECTIONS

(Org of SECTION (name Organization) (total-

questions 25) (positives 15)

… (rest of the instances))

- Abstract: This knowledge base contains the

abstraction rules required for reaching the level

of compliance with each dimension in the

desired KPQM level. A rule example is as

follows:

(defrule is-organization-none

(object (name [Org]) (test (< positives (* total-

questions 0.25))) =>

(send [Org] put-category none))

- Level2, Level3, Level4, and Level5: This

knowledge base contains the rules for the

evaluation of the criteria Last-level and

Concrete-level. A rule example for Level 2 is as

follows:

(defrule level2-ok-1

(object (name [Org]) (category high))

(object (name [Peop]) (category medium))

(object (name [Tech]) (category medium)) =>

(send [ConLevel] put-truth_value true))

- Decision: These rules refer to the KBS final

decision according to the values of the criteria

specified above. A rule example is as follows:

(defrule non-possible-1

(object (name [ConLevel]) (truth_value false))

(object (name [LasLevel]) (truth_value false))

(send [LevDec] put-value false))

The Clips inference engine is started, the

corresponding knowledge bases are loaded and the

inferential process begins. Figure 7 shows an

execution example in which an organization tries to

be evaluated at level 4 after having been

successfully audited at level 3. In this case the

organization lacks an acceptable level of compliance

with the defined dimensions.

4 CONCLUSIONS

In the current competitive context the effective KM

DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL EVALUATION SYSTEM USING COMMONKADS

463

Figure 7: An execution example.

implies prestige and competitive advantages for the

organizations. The evaluation of the quality of the

KM initiatives is precisely the purpose of the KM

maturity models. However, usually for reaching the

desired KM maturity level an organization needs to

overcome a series of expensive audits. This paper

proposes a KBS to reduce these costs by limiting the

audits to those cases in which the KBS output is

positive; that is, the system considers that the

organization complies with the desired KM level.

The proposed KBS implements the evaluation

following the KPQM model.

It should be highlighted that the developed KBS

is currently being installed and tested in various

companies at A Coruña, Spain, with which the

authors have previously collaborated. Several test

batteries have been run with virtual and real data in

order to validate the system. At the moment, one of

the organizations has been successfully evaluated in

KPQM level 2 by the system and we are waiting for

the auditor’s decision in order to compare both

results and improve the system if necessary.

REFERENCES

Kuriakose, K. K., Raj, B., Satya Murty, S. A. V.,

Swaminathan P., 2010. Knowledge Management

Maturity Models – A Morphological Analysis. Journal

of Knowledge Management Practice, 11 (3).

Davenport, H., Prusak, L., 2000. Working Knowledge.

Harvard Business Press.

Ares J., Pazos J., 1998. Conceptual Modeling: an Essential

Pillar for Quality Software Development. Knowledge-

Based Systems, 11, pp. 87-104.

Gallangher, S., Hazlett, S., 2004. Using the Knowledge

Management Maturity Model (KM3) As an Evaluation

Tool. URL: http://cc.shu.edu.tw/~yjliu/%AA%BE%C

3%D1%BA%DE%B2z/%B0%D1%A6%D2%BE%5C

%C5%AA%B8%EA%AE%C6/km028.pdf

KPMG, 2000. Knowledge Management Research Report.

URL:http://www.providersedge.com/docs/km_articles

/KPMG_KM_Research_Report_2000.pdf

Klimko, G., 2001. Knowledge Management and Maturity

Models: Building Common Understanding. In

Proceeding of the 2nd European Conference on

Knowledge Management, pp. 269-278.

Kulkarni, U., Freeze, R., 2004. Development and

Validation of a Knowledge Management Capability

Assessment Model. In Proceeding of Twenty fifth

International Conference on Information Systems, pp.

657-670

Paulzen, O., Perc, P., 2002. A Maturity Model for Quality

Improvement in Knowledge Management. In

Australasian Conference on Information Systems, pp

243-253.

Kochikar, V. P., 2000. The Knowledge Management

Maturity Model: A Staged Framework for Leveraging

Knowledge, KM World, Santa Clara, CA.

McCollum, W. R., 2006. Process Improvement in Quality

Management Systems: A Case Study Analyzing

Carnegie Mellon's Capability Maturity Model (CMM),

Trafford Publishing.

Persse, J. R., 2001. Implementing the Capability Maturity

Mode, Wiley.

SEI, 2001. Software Engineering Institute homepage.

URL: www.sei.cmu.edu/

SPICE, 2011. SPICE homepage. URL:http://www.sqi.

gu.edu.au/spice/

CommonKADS, 2011. The CommonKADS homepage.

URL: http://www.commonkads.uva.nl/

Schreiber, G., de Hoog, R., Akkermans, H., Anjewierden,

A., Shadbolt, N., de Velde, W. V, 2000. Knowledge

Engineering and Management: The CommonKADS

Methodology, The MIT Press.

Kingston, J., 1998. Designing Knowledge Based Systems:

The CommonKADS Design Model. Knowledge-Based

Systems, 11 (5-6), pp. 311-319.

Valente, A., Breuker, J, van de Velde, W., 1998. The

CommonKADS Library in Perspective. International

Journal of Human-Computer Studies, 49(4), pp. 391-

416.

Clips, 2011, The Clips tool homepage. URL. http://clipsru

les.source forge.net/

ICAART 2012 - International Conference on Agents and Artificial Intelligence

464