DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL
EVALUATION SYSTEM USING COMMONKADS
Javier Andrade, Juan Ares, Rafael García, Santiago Rodríguez and Sonia Suárez
Software Engineering Laboratory, University of A Coruña, Campus de Elviña s/n,15071, A Coruña, Spain
Keywords: CommonKADS, Clips, Knowledge-Based System, Knowledge Management, KPQM.
Abstract: Several knowledge management maturity models have been proposed in the last years. These models are
used to evaluate the quality of knowledge management practices in the organizations. One of these models
is the Knowledge Process Quality Model, which has five maturity levels. The acquisition of a high maturity
level is usually expensive due to the evaluations and improvement processes that are often required for a
positive final decision. With the aim of minimizing these costs, this paper proposes a Knowledge-Based
System that tries to check if the company currently stands in compliance with a given KPQM maturity level.
The actual evaluation process starts only if the system output is positive. This approach implies an important
cost reduction by avoiding negative evaluations. The design of the system is based on the CommonKADS
methodology, and its implementation was carried out with the Clips tool.
1 INTRODUCTION
According to Kuriakose et al. (2011), Knowledge
Management (KM) is a discipline that tries to create
wealth and value by providing the right knowledge
at the right place and at the right time. The effective
use of knowledge by several organizational entities
results in improved skills and competencies for
decision making, performance improvement and
also innovation. Thus, the organizations had devoted
numerous efforts for retaining and institutionalizing
the knowledge they possess (Davenport and Prusak,
2000). In this way, there are several KM initiatives
in the literature (Ares et al., 2008) that try to
establish guides for effectively implementing and
developing the KM in the organizations.
The following obvious step involves the set up of
mechanisms for assessing how well these KM
initiatives are deployed in the organizations. This
assessment focuses on the quality of the organization
processes, as well as in the mechanisms used for
achieving and keeping these processes. The later
mechanisms should also be used as a guide for
continuous improvement.
As a result, there is a growing number of the so-
called Knowledge Management Maturity Models
(KMMM). Examples of these initiatives are: the
KMf and KM
3
models (Gallagher and Hazlett,
2000), the KPMG Knowledge Journey (KPMG,
2000), the model proposed by Gabor Klimko (2001),
the Knowledge Management Capability Assessment
(KMCA) (Kulkarni and Freeze, 2004; 2005), the
Knowledge Process Quality Model (KPQM)
(Paulzen and Perc, 2002) and the model proposed by
Infosys (Kochikar, 2000).
The compliance with a high KM maturity level
increases the prestige and competitiveness of the
organizations. However, the evaluation process is
often quite expensive. An auditor has to determine
the compliance with a given KM maturity level. In
the case of non compliance, the defects detected
have to be solved and a new audit is required to the
evaluation of the organization in the desired KM
maturity level.
This paper proposes a Knowledge-Based System
(KBS) for the evaluation of an organisation in a KM
maturity level. This KBS will return a positive or
negative compliance report with regards to the given
KM maturity level. The services of the auditor are
required only when the result of the KBS is positive.
There is a high matching probability between
this report and that of the auditor, but sometimes this
matching may not happen, since the auditor may
consider certain aspects that are not considered by
the KBS. These aspects are also useful, since the
inclusion of new knowledge will improve the KBS.
The KPQM model, proposed by Paulzen and
Perc (2002), was selected because its structure and
459
Andrade J., Ares J., García R., Rodríguez S. and Suárez S..
DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL EVALUATION SYSTEM USING COMMONKADS.
DOI: 10.5220/0003697504590464
In Proceedings of the 4th International Conference on Agents and Artificial Intelligence (ICAART-2012), pages 459-464
ISBN: 978-989-8425-95-9
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
processes are well detailed. This model is based on
the ideas of two software development maturity
models: the Capability Maturity Model (CMM)
(McCollum, 2006) (Persse, 2001) (SEI, 2011) and
the Software Process Improvement and Capability
dEtermination (SPICE) (SPICE, 2011).
The KPQM model description and the design of
the proposed KBS are detailed in Section 2; Section
3 shortly describes the KBS implementation, and
Section 4 sets out the conclusions.
2 DESIGN OF THE PROPOSED
SYSTEM
2.1 KPQM Description
Starting from the CMM and SPICE models, Paulzen
and Perc (2002) developed a model that allows the
assessment of KM processes in an organization. The
model also provides a summary of the steps to be
followed in order to reach some improvements. The
model includes five maturity levels (see Table 1).
Table 1: Maturity levels.
Maturity Level Description
1. Initial
The quality of the knowledge
processes is not defined and it
randomly changes. This level
could be described as chaotic.
2. Aware
The need of managing
knowledge processes starts to be
borne in mind. The first
structures for guarantying a better
quality of processes are
implemented.
3. Established
A systematic structure and a
definition for knowledge
processes are established.
4. Quantitatively
Manager
The systematic management of
processes is emphasized by
means of performing output
measurements for their planning
and guidance.
5. Optimizing
There are structures for
continuous improvement.
Each maturity level defines a series of issues
with which the organization must comply. That
structure, as Table 2 shows, rests on the following
dimensions: Organization, People and Technology.
The KBS design has to reflect that structure in
order to assess the compliance with all the maturity
levels defined in the KPQM.
Table 2: Assessment structure.
Level Dimension Issue
1
Organization
-
People
-
Technology
-
2
Organization
Defined and documented
processes.
Process-native basic abilities
and structures.
People
Structures for the employees
to consider KM processes.
Structures for the direction
to consider KM processes.
Technology
It includes part of the
technological support for
KM methods.
3
Organization
Process standardization.
Structured knowledge.
People
There is an incentive system
for employees in order to
boost the use of knowledge.
There is an incentive system
for executives in order to
boost the use of knowledge.
Technology
There is a systematic
technological support for the
processes.
4
Organization
The processes are
quantitatively assessed.
The management decisions
are quantitatively assessed.
People
The incentive system for
employees is quantitatively
assessed.
The incentive system for
executives is quantitatively
assessed.
Technology
The impact of the
technological support is
quantitatively assessed.
5
Organization
There are structures for
improving the processes.
There are structures for
improving the management.
People
There are structures that
promote the continuous
improvement in the use of
knowledge.
There are structures that
guarantee that the
management might keep
involved in the KM tasks.
Technology
The process-supporting
technologies are optimized.
Pilot projects are carried out.
The quality of the KBS design depends on the
programming skills of the knowledge engineers and
also on their abilities to devise, remember, and
ICAART 2012 - International Conference on Agents and Artificial Intelligence
460
dynamically update a design specification. This is a
difficult task for all but the smallest KBSs.
Difficulties like these can be alleviated by
producing textual representations or diagrams of
expert knowledge and design specifications. The
best known approach for producing such documents
is the CommonKADS methodology
(CommonKADS, 2011) (Schreiber, 2000)
(Kingston, 1998) (Valente, 1998). CommonKADS is
the current European de facto standard for
knowledge analysis and knowledge-intensive
systems development. This methodology has been
wholly or partially incorporated to existing methods
by many major companies in Europe, US and Japan
(CommonKADS, 2011). CommonKADS was used
in this work for elaborating a list of potential model
components for the KBS, selecting the adequate task
template, constructing the initial domain scheme,
and completing the specification of the knowledge
model. The following sections describe how these
activities were carried out.
2.2 List of Potential Model
Components
The task of the proposed KBS belongs to a highly
specialized field (a concrete and classified theme
within Quality Management). Mainly due to this
reason there is information available on how to carry
out audits (SEI, 2011). Consequently, the knowledge
of the domain can be considered as formal.
On the one hand, there is evidence of the
existence of a commonly accepted structure in the
KPQM model—shown in Figure 1—that represents
an initial candidate for the domain model. This
structure reflects the existence of five maturity levels
and three common dimensions. Each dimension is
related to a set of issues in each maturity level.
Also, a maturity level requires a certain level of
compliance with each dimension, and each
dimension contains a series of issues presented by
maturity-shaped questions. We have been able to
define these questions thanks to the KPQM literature
and the information available on how to carry out
audits (c.f., e.g., SEI, 2011).
Figure 1: Initial relationships structure.
On the other hand, it is essential to record the
performed audits and their results (e.g. in a file or
database). Therefore, before resolving the question
about a KM maturity level n, the KBS has to check
if the organization satisfies the KM maturity level n-
1. This practice follows the Quality Management
philosophy, which stands on not skipping maturity
levels. In that way, level 2 should be previously
achieved in order to reach level 3.
2.3 Selection of the Task Template
The aim of the proposed KBS is to fill a form with
organizational information and analyze it in order to
determine if the organization currently stands in
compliance with the selected KPQM maturity level;
that is, if an actual evaluation is possible.
In this context, and from the point of view of the
task, this is an activity that fits into the category of
assessment. These activities are provided with
various templates, from which we have selected the
one mentioned in Schreiber et al. (2000).
The main motive for this choice is that the
associated inferential structure matches the purpose
of the KBS. A good technique to establish this
adequacy is building an annotated inferential
structure in which the roles are annotated with
specific elements of the domain. This inferential
structure is shown in Figure 2.
Figure 2: Annotated inferential structure.
2.4 Construction of the Initial Domain
Scheme
The result of this task is a set of domain-specific
conceptualizations — shown in Figure 3 — and a set
of method-specific conceptualizations — shown in
Figure 4.
Two main concepts were detected in the domain:
Form and Section. The information about the last
KPQM level reached by the organization was also
required and it is represented by the concept Record.
DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL EVALUATION SYSTEM USING COMMONKADS
461
Figure 3: Domain-specific conceptualizations.
Figure 4: Method-specific conceptualizations.
The Form and Record concepts constitute the
initial reasoning case. A Form refers to a specific
KPQM level and consists of three sections each of
them representing a dimension (Organization,
People, and Technology). There is an aggregation
relationship between the concepts Form and Section.
The concept Form has an attribute associated-level
that indicates the desired KPQM level. The concept
Section has four attributes: name, total-questions,
positives, and category. The first refers to the name
of the dimension—e.g. People—, the second
indicates the total number of issue-related questions
in the dimension, the third represents the total
number of positively answered questions, and the
last attribute refers to the level of compliance with
the section. This level of compliance is obtained by
means of the total-questions and positives attributes
as follows:
- If the positive answers represent less than 25%
of the total, the level of compliance is none: the
organization does not comply with the
dimension represented by the section.
- If the positive answers represent between 25-
50% of the total, the level of compliance is low.
- If the positive answers represent between 50-
75% of the total, the level of compliance is
medium.
- If the positive answers represent between 75-
100% of the total, the level of compliance is
high.
Once it is determined how the domain concepts
will be used, there should be settled the criteria to be
applied to the data in order to determine the
compliance with a given KPQM level. In this case,
two different criteria are considered, each with a
attribute truth-value representing whether or not the
criterion is fulfilled:
- Last-level: Was the organization successfully
audited in the level n-1?
- Concrete-level: Does the organization meet the
requirements of the level n? The organization
must meet the issues in the level dimensions at
certain rates or levels (many possibilities are
accepted).
Finally, it should be highlighted that the system
only offers a positive answer if all the criteria have
the true value.
2.5 Complete Specification of the
Knowledge Model
As explained before, the activity to be modelled is
an example of the task type assessment. Also, the
selected template shows an adequate inferential
structure for the purpose of this KBS, in which the
inferences present sufficient detail.
The task that must be carried out is decomposed
into two subtasks, which means that the task method
structures the reasoning process in two steps:
- Abstraction: the purpose of this step is to achieve
the compliance level for each section. As
explained above, this level can be none, low,
medium or high. For the KBS to reason as an
expert does, the meaning of the number of the
positive answers should be known. That is, the
reasoning of an expert auditor may be: The
organization complies with all the dimensions at
a medium level, but the People dimension must
have a high level of compliance in this KPQM
maturity level and I therefore consider that
improvements must be made in that dimension.
- Matching: the abstractions are matched in order
to take the final decision on whether or not the
organization complies with the established
criteria.
Figure 5 shows the template that was chosen for
the modelling.
On the other hand, the knowledge scheme that
was finally obtained is shown in Figure 6. It can be
observed that the final domain scheme incorporates
three rule types:
- Case-abstraction: abstraction rules required for
reaching the section compliance level by using
the attributes total-questions and positives.
ICAART 2012 - International Conference on Agents and Artificial Intelligence
462
assess case
assess through
abstract & match
abstract case match case
abstract
method
match
method
specify select evaluate match
task
task method
task
task method
infe rence
assess case
assess through
abstract & match
abstract case match case
abstract
method
match
method
specify select evaluate match
task
task method
task
task method
infe rence
Figure 5: Decomposition of the task.
- Form-requirement: rules offering truth values
to the criteria Last-level and Concrete-level.
The first one indicates the compliance with the
previous KPQM level and the second one
indicates the acceptable compliance levels for
the desired KPQM level.
- Level-decision-rule: The decision is represented
by a concept Level-decision with an attribute
that indicates whether or not the organization
has real possibilities of successfully
overcoming an audit for the desired KPQM
level. Also, this rule expresses the relationship
between the different criteria and the final
decision taken by the KBS.
form-requirement
level-decision-rule
case-abstraction
Case
indicates
has abstraction
KPQMLevel Criterion
implies
level-decision
form-requirement
level-decision-rule
case-abstraction
Case
indicates
has abstraction
KPQMLevel Criterion
implies
level-decision
Figure 6: Final knowledge scheme.
3 IMPLEMENTATION OF THE
PROPOSED SYSTEM
The system was implemented according to the
CommonKADS design and by means of the Clips
tool (Clips, 2011). In order to provide the
application with modularity and to simplify the
development and depuration processes, the
following knowledge bases were defined:
- General: This knowledge base contains all the
definitions of classes, objects, and properties. It
contains the operative knowledge of the system.
See for example, the concept Section definition:
(defclass SECTION (is-a USER)
(role concrete)
(slot name (type STRING))
(slot total-questions (type INTEGER))
(slot positives (type INTEGER))
(slot category (allowed-values none low medium
high) (default none))
and the object Org (KPQM dimension
Organization) definition:
(definstances SECTIONS
(Org of SECTION (name Organization) (total-
questions 25) (positives 15)
… (rest of the instances))
- Abstract: This knowledge base contains the
abstraction rules required for reaching the level
of compliance with each dimension in the
desired KPQM level. A rule example is as
follows:
(defrule is-organization-none
(object (name [Org]) (test (< positives (* total-
questions 0.25))) =>
(send [Org] put-category none))
- Level2, Level3, Level4, and Level5: This
knowledge base contains the rules for the
evaluation of the criteria Last-level and
Concrete-level. A rule example for Level 2 is as
follows:
(defrule level2-ok-1
(object (name [Org]) (category high))
(object (name [Peop]) (category medium))
(object (name [Tech]) (category medium)) =>
(send [ConLevel] put-truth_value true))
- Decision: These rules refer to the KBS final
decision according to the values of the criteria
specified above. A rule example is as follows:
(defrule non-possible-1
(object (name [ConLevel]) (truth_value false))
(object (name [LasLevel]) (truth_value false))
=>
(send [LevDec] put-value false))
The Clips inference engine is started, the
corresponding knowledge bases are loaded and the
inferential process begins. Figure 7 shows an
execution example in which an organization tries to
be evaluated at level 4 after having been
successfully audited at level 3. In this case the
organization lacks an acceptable level of compliance
with the defined dimensions.
4 CONCLUSIONS
In the current competitive context the effective KM
DEVELOPING A KNOWLEDGE PROCESS QUALITY MODEL EVALUATION SYSTEM USING COMMONKADS
463
Figure 7: An execution example.
implies prestige and competitive advantages for the
organizations. The evaluation of the quality of the
KM initiatives is precisely the purpose of the KM
maturity models. However, usually for reaching the
desired KM maturity level an organization needs to
overcome a series of expensive audits. This paper
proposes a KBS to reduce these costs by limiting the
audits to those cases in which the KBS output is
positive; that is, the system considers that the
organization complies with the desired KM level.
The proposed KBS implements the evaluation
following the KPQM model.
It should be highlighted that the developed KBS
is currently being installed and tested in various
companies at A Coruña, Spain, with which the
authors have previously collaborated. Several test
batteries have been run with virtual and real data in
order to validate the system. At the moment, one of
the organizations has been successfully evaluated in
KPQM level 2 by the system and we are waiting for
the auditor’s decision in order to compare both
results and improve the system if necessary.
REFERENCES
Kuriakose, K. K., Raj, B., Satya Murty, S. A. V.,
Swaminathan P., 2010. Knowledge Management
Maturity Models – A Morphological Analysis. Journal
of Knowledge Management Practice, 11 (3).
Davenport, H., Prusak, L., 2000. Working Knowledge.
Harvard Business Press.
Ares J., Pazos J., 1998. Conceptual Modeling: an Essential
Pillar for Quality Software Development. Knowledge-
Based Systems, 11, pp. 87-104.
Gallangher, S., Hazlett, S., 2004. Using the Knowledge
Management Maturity Model (KM3) As an Evaluation
Tool. URL: http://cc.shu.edu.tw/~yjliu/%AA%BE%C
3%D1%BA%DE%B2z/%B0%D1%A6%D2%BE%5C
%C5%AA%B8%EA%AE%C6/km028.pdf
KPMG, 2000. Knowledge Management Research Report.
URL:http://www.providersedge.com/docs/km_articles
/KPMG_KM_Research_Report_2000.pdf
Klimko, G., 2001. Knowledge Management and Maturity
Models: Building Common Understanding. In
Proceeding of the 2nd European Conference on
Knowledge Management, pp. 269-278.
Kulkarni, U., Freeze, R., 2004. Development and
Validation of a Knowledge Management Capability
Assessment Model. In Proceeding of Twenty fifth
International Conference on Information Systems, pp.
657-670
Paulzen, O., Perc, P., 2002. A Maturity Model for Quality
Improvement in Knowledge Management. In
Australasian Conference on Information Systems, pp
243-253.
Kochikar, V. P., 2000. The Knowledge Management
Maturity Model: A Staged Framework for Leveraging
Knowledge, KM World, Santa Clara, CA.
McCollum, W. R., 2006. Process Improvement in Quality
Management Systems: A Case Study Analyzing
Carnegie Mellon's Capability Maturity Model (CMM),
Trafford Publishing.
Persse, J. R., 2001. Implementing the Capability Maturity
Mode, Wiley.
SEI, 2001. Software Engineering Institute homepage.
URL: www.sei.cmu.edu/
SPICE, 2011. SPICE homepage. URL:http://www.sqi.
gu.edu.au/spice/
CommonKADS, 2011. The CommonKADS homepage.
URL: http://www.commonkads.uva.nl/
Schreiber, G., de Hoog, R., Akkermans, H., Anjewierden,
A., Shadbolt, N., de Velde, W. V, 2000. Knowledge
Engineering and Management: The CommonKADS
Methodology, The MIT Press.
Kingston, J., 1998. Designing Knowledge Based Systems:
The CommonKADS Design Model. Knowledge-Based
Systems, 11 (5-6), pp. 311-319.
Valente, A., Breuker, J, van de Velde, W., 1998. The
CommonKADS Library in Perspective. International
Journal of Human-Computer Studies, 49(4), pp. 391-
416.
Clips, 2011, The Clips tool homepage. URL. http://clipsru
les.source forge.net/
ICAART 2012 - International Conference on Agents and Artificial Intelligence
464