When Measuring Performance Makes the Difference
Quality and Success of a Clinical Knowledge Management Project
Brigitte Stroetmann, Alena Leinfelder, Alessandro Ortisi and Okan Ekinci
Siemens AG Healthcare Sector, Allee am Röthelheimpark 3 A, Erlangen, Germany
Keywords: Knowledge Management, Performance Measurement, User Satisfaction Survey, Knowledge Services,
Learning Organization.
Abstract: This paper is designed mainly to provide a picture of how the value of Knowledge Management Projects
within Siemens Healthcare is assessed and analyzed. Due to the growing number of knowledge and new
research areas, especially in medicine, it is essential to provide up to- date information with an impact on
the healthcare business. The degree to which the organization effectively uses this information is an
indicator of the organizational knowledge management maturity. The main question to be answered is
“Which performance indicators should be measured when analyzing the quality and success of a knowledge
management project”. We reviewed available literature to validate the performance indicators that show
positive outcomes of the projects. Most importantly we want to emphasize how easily these metrics can be
implemented into the company’s operational business. We, therefore, recommend web-based surveys and
reporting tools that automatically measure and calculate the results. The information should be easy to read
and enable effortless performance change monitoring.
1 INTRODUCTION
Siemens Healthcare brings together innovative
imaging and laboratory diagnostic equipment,
information technology, management consulting and
services to help customers achieve positive and
sustainable clinical and financial outcomes. The
company focuses on strategies to increase both
efficiencies and quality of care, while
simultaneously reducing costs. Its scientists are
professionals who focus on driving innovation for
customer excellence. Managing knowledge has
turned out to be a key to initiating successful
business, as the healthcare environment is
continuously changing and becoming increasingly
competitive.
Thus Siemens Healthcare follows an active
knowledge management strategy. As most of the
employees have a scientific, technical, strategic or
financial background, but not necessarily deep
medical knowledge, the company decided to provide
their employees with current, up-to-date medical
knowledge that is of relevance for innovations in
solution and product development. The clinical
knowledge management approach includes the
gathering, evaluation, storage and use of relevant
medical information. It is based on three pillars: the
Clinical Competence Centers, a group of medical in-
house experts, who give medical advice on
individual questions, the Clinical Knowledge Base –
a knowledge repository for disease-specific
information, and finally the Siemens Healthcare
Academy
By measuring the performance of activities in the
Knowledge Management project it is possible to
ajust the services to changing demand patterns and
business strategies.
2 BASIC CONCEPTS
Knowledge Management has been around for more
than 20 years and there are dozens of strategies,
tools and approaches for making the most of
knowledge assets in an organization. The ever rising
awareness regarding the impact of successful KM
strategies results from the fact that knowledge assets
represent the fount of a company’s competences that
are deemed essential for customer satisfaction,
competitive advantage and product innovation.
Companies are undergoing fundamental changes as
the emphasis has steadily moved from physical or
419
Stroetmann B., Leinfelder A., Ortisi A. and Ekinci O..
When Measuring Performance Makes the Difference - Quality and Success of a Clinical Knowledge Management Project.
DOI: 10.5220/0004616704190425
In Proceedings of the International Conference on Knowledge Discovery and Information Retrieval and the International Conference on Knowledge
Management and Information Sharing (KMIS-2013), pages 419-425
ISBN: 978-989-8565-75-4
Copyright
c
2013 SCITEPRESS (Science and Technology Publications, Lda.)
Table 1: Literature Overview.
Author Title Main Issue Indicators
Bhatti et al.
(2011)
The effect of knowledge
management practices on
organizational performance: A
conceptual study
KM success via a
knowledge sharing
culture
process, intellectual capital, culture and
strategy
Davenport et
al. (1998)
Successful Knowledge
Management Projects
KM project success
via identifying key
characteristics
economic performance, infrastructure,
balanced structure, positive environment,
clear purpose and terminology, motivation,
multiple channels, management support
DeLone and
Mc Lean
(2003/1992)
The DeLone and McLean
Model of Information Systems
Success: A Ten-Year Update
KM success via a
multidimensional
and interdependent
construct
system quality, information quality, use,
user satisfaction, individual impact,
organizational impact
Jadoon and
Hasnu (2009)
Collaboration Dichotomies in
Knowledge Management
Success
KM success needs
intra-organizational
collaborations
intra-community social interactions, KM
systems
Jennex et al.
(2008)
Towards a consensus
knowledge management
success definition
definition of
knowledge
management success
impact on business processes, strategy,
leadership, and knowledge content.
Jennex et al.
(2012)
Where to look for Knowledge
Management Success
KM success as a
multidimensional
concept
impact on business processes, strategy,
leadership, and knowledge content.
tangible assets to intangible knowledge assets. This
has important implications for how knowledge
assets are acquired, sourced, created and utilized,
and how the dissemination of knowledge and
effectiveness of knowledge assets can be measured.
The literature is packed with recommendations
and also controversial discussions about KM
performance measurements.
A section of the literature argues strongly that the
development of KM performance metrics is crucial:
without metrics, knowledge cannot be produced.
Without measurement, knowledge management
results into a metaphysical exercise with little
actionable value. Metrics are important because
what gets measured gets done. According to the
American Quality and Productivity Center (APQC,
2003) knowledge is an intangible asset, but the
impact of KM is measurable.
Other authors doubt the value of KM
performance measurement ‘It is certainly possible to
identify metrics … The best known of these was
developed by financial services group Skandia, but
there are some doubts as to whether it provides a
generally portable model’ (Gamble and Blackwell,
2001). Unfortunately there is no direct relationship
between an intangible asset and financial outcome.
Thus, financial accounting and traditional
accounting instruments fail to capture these values
and report them.
A review of the KM performance literature
reveals that there is no certain set of performance
metrics that fits all organizations and is as well
recommended by the KM community.
From a knowledge perspective, “Knowledge
Management success is a multidimensional concept”
(Jennex, 2008) which has different interacting
components. Jennex et al. measured KM success by
means of the following dimensions: impact on
business processes, impact on strategy, leadership,
and knowledge content.
To verify these components, in 2012 Jennex et
al. published the results of their KM success survey
and showed that the more successful a knowledge
management initiative the more the KM measured
items in more dimensions. This confirms the
multidimensional approach model and shows that
successful knowledge managers should use multiple
measurements in all four dimensions.
This multidimensional concept was adapted from
De Lone and Mc Lean (2003/1992) who identified
the six dimensions: system quality, information
quality, use, user satisfaction, individual impact and
organizational impact. Information system success is
therefore a multidimensional and interdependent
construct with important interrelationships. This
approach is one of the basic models for KM success
and was modified by many authors such as Jennex et
al.
KMIS2013-InternationalConferenceonKnowledgeManagementandInformationSharing
420
A basic approach to identifying the main
objectives of KM projects was adopted by
Davenport et al. (1998). They identified four main
goals: create knowledge repositories, improve
knowledge access, enhance knowledge environment
and manage knowledge as an asset. For knowledge
repositories it is necessary to capture the information
and store it, for example by means of an IT system
which can be easily accessed. Improving knowledge
access does not only mean access to a database but
also connecting knowledge holders and knowledge
recipients in an effective way. This leads directly to
the creation of a knowledge environment which can
be turned into a core asset for an organization. To
measure whether the knowledge initiatives are
successful, Davenport et al. identified eight key
findings such as technical and organizational
infrastructure, knowledge-friendly culture, clear
purpose and terminology, multiple channels for
knowledge transfer, senior management support etc.
In particular the aspect of a knowledge friendly
environment with the opportunity to collaborate and
share knowledge is addressed by many other
authors, too.
Bhatti et al. (2011) performed a literature
screening, and defined KM success via a sharing
culture between the employees. They therefore
developed a conceptual frame work model including
processes, intellectual capital, culture and strategy,
called PICS, as pillars for organizational KM. For
successful KM it is necessary to create a strategy
which considers the whole of the knowledge a
company offers and turn it into a sustainable core
competency.
Jadoon and Hasnu (2009) agree with the
principle of a knowledge sharing culture and also
claim an IT concept focusing on intra-
organizational collaborations. They were able to
show that there really is a significant positive
correlation between KM systems and collaboration
and the resulting success.
Much has been written about KM performance
measurements, but little of it provides practical
methods to measure the KM state of the
organization.
The challenges involved in the precise
measurement of complex interrelations can be seen
from the sign that hung in Albert Einstein’s office in
Princeton: “Not everything that can be counted
counts, and not everything that counts can be
counted”. Subsequently we have been looking for
performance indicators that may not necessarily
show the improving overall organizational
performance, but do show if the knowledge activity
is improving / increasing or not. These determine the
status of the KM project and whether it has
established a level of satisfaction or if there is a need
for some improvement actions (Robertson, 2003).
3 SIEMENS HEALTHCARE
CLINICAL KNOWLEDGE
MANAGEMENT PROJECT
In order to understand what is going to be measured,
this section briefly describes the main elements of
the Clinical Knowledge Management Project.
3.1 Purpose of Implementing a Clinical
Knowledge Management Project
The Siemens Healthcare Sector is one of the world’s
largest suppliers to the healthcare industry and also a
trendsetter in medical imaging, laboratory
diagnostics, medical information technology and
hearing aids. Siemens offers its customers products
and solutions for the entire range of patient care
from a single source – from prevention and early
detection to diagnosis, and on to treatment and
aftercare. The company focuses on intangible assets
as they have a greater impact on the value of a high
technology sector, such as health care, than on other
sectors.
As such, the management of knowledge is
paramount in the context of health care industry
organization. Current medical information and
knowledge are some of the core assets in the health
care industry, but information overload is a real
challenge. Medical knowledge has been expanding
exponentially during the last two or three decades,
and new areas of research, such as
bionanotechnology and genetics, are growing at a
tremendous pace. Moreover, most of the employees
have a scientific engineering, strategic or financial
background, but not necessarily deep medical
expertise. Thus medical knowledge has to be
actively managed.
3.2 Active Knowledge Management
Approach
Siemens Healthcare takes an active approach to
medical KM by executing a series of strategies to
improve the way knowledge is managed.
A dedicated team of KM “workers”, who are
actively managing the operational KM process, co-
WhenMeasuringPerformanceMakestheDifference-QualityandSuccessofaClinicalKnowledgeManagementProject
421
ordinates the basic processes of the knowledge
management life-cycle. This comprises
identification of medical knowledge needs, the
creation and dissemination of medical information
as well as storage and updating. As the expertise of
most employees is related to economics, business, or
technical issues, the team actively manages the
timely dissemination of relevant medical
information into the organization. The KM program
offers three services. First, pull services (self-
service) – contribute & retrieve knowledge as and
when it is needed. Second, push services (facilitated
transfer of knowledge) driven by the needs of the
organization. And third, individual expert advice
where peers and experts join forces to discuss and
share knowledge.
3.2.1 Clinical Competence Center - The
Central Source of Medical Knowledge
Highly regarded physicians from the fields of
cardiovascular medicine, oncology and
neurosciences are working closely together in the
Clinical Competence Centers (CCC).
They provide in-depth medical information and
educational training required to fully understand the
customer’s needs in various units of the company,
from marketing to sales and from product definition
to R&D. They also perform a regular scientific
literature review, summarize and record the content
of each, and provide a critical review of the major
works to identify the potential impact of scientific
findings on Siemens Healthcare business and put
strategic decisions in marketing and product
development on a sound basis.. Furthermore, the
members of the Clinical Competence Centers build
and maintain networks of key opinion leaders in
their specialties to identify future disease trends at
any early stage.
Although the physicians in the CCCs have broad
practical experience in the diagnosis and treatment
of diseases, standard of care evolve, clinical
pathways and decision making as well as procedures
change over time. . Thus, the physicians of the
CCCs also practice in hospitals to keep up with
medical progress and advances within their
specialties.
3.2.2 the Clinical Knowledge Base - The
Online Portal for Healthcare
Siemens Healthcare operates the knowledge sharing
platform Clinical Knowledge Base on a Microsoft
Sharepoint® 2007 Business & Collaboration
Platform. It connects the 40,000 Healthcare Sector
employees worldwide and allows online exchange of
up-to-date medical information. The Clinical
Knowledge Base features up-to-date information,
research insights, technology trends, news and
reports that might be relevant to the company’s
current and future product development activities
(Fig.1).
Every single document has been commented on
and reviewed by a member of the CCCs. One of the
great benefits the users of the Clinical Knowledge
Base experience is the worldwide access to the
information pool. The Clinical Knowledge Base
offers an intuitive user interface and flexible
workflow experience that supports individual
problem-solving approaches. The platform also
provides a space to post urgent requests via a simple
web-based interface. Here, the experts of the CCCs
offer support for individual projects and inquiries.
Figure 1: Clinical Knowledge Base.
3.2.3 Siemens Healthcare Academy-
Life-long Learning Possibilities for
Siemens Employees
Siemens Healthcare Academy is an international
initiative across Healthcare. The objective is to help
all Healthcare employees understand their
customers’ clinical workflows and, therefore, their
requirements. Only those who know their customers
well can offer solutions tailored to their daily work
environment.
The first component was a multimedia, web-
based learning program that tackled the subject of
KMIS2013-InternationalConferenceonKnowledgeManagementandInformationSharing
422
clinical workflow. The current offering comprises
clinical basics and advanced courses as well as
dedicated sales training. In addition to classroom
trainings, a variety of e-learning courses are
available. Blended learning concepts are applied to
optimally use resources. For instance a web-based
training on basic cardiology is a prerequisite for
more advanced classroom trainings in cardiology.
Thus, the curriculum and content of both e-learning
and classroom trainings are interlinked.
4 PERFORMANCE INDICATORS
AND MEASUREMENT
“According to a study by APQC on measuring the
impact of knowledge management, many companies
simply rely on their standard performance
measurement of items such as growth, profit, and
new product sales to evaluate the impact of
knowledge management. While these types of
outcome measurements are certainly important, they
are highly unlikely to be influenced solely by
knowledge management activities, and therefore
make poor metrics. A further problem is that these
outcome measurements are all lagging indicators,
and it might take several years for the sharing of
knowledge to show up in a bottom line performance
measurement” (Brown).
Thus we pragmatically looked at performance
metrics or key performance indicators that provide a
practical way to measure the activity of the KM-
project and their impact. “The most important
characteristic to consider when choosing or defining
a way of measuring KM performance is whether the
metric indicates if knowledge is being shared and
used” (Hoss and Schlussel, 2009). Also, good
metrics are reliable, repeatable, easy to use and
consistent. “When a content management system is
in place, a suitable metric could be the number of
documents downloaded from the repository. While
this is a numeric metric, metrics can also be
qualitative, e.g. improved employee satisfaction.
The performance of the project is measured in
active involvement. Examples of indicators used are
the number of participating employees; the number
of requests and postings as well as the number of
downloads from the Clinical Knowledge Base. Other
measurements include end-user satisfaction with
support from the experts of the Clinical Competence
Centers, and end-user satisfaction with the
cooperation with the Knowledge Management
Team.
4.1 Automate Measurement-
Webtrends as Reporting Tool
When possible and reasonable, we tried to build the
performance metrics into the design of the Clinical
Knowledge Base itself; thus the metrics are
automatically generated during normal usage of the
Knowledge Base. Some years ago Siemens Sector
Healthcare introduced “webtrends” software
solution offerings to continuously measure web
traffic on the intranet, and also to analyze and assess
its website.
Webtrends describes the use of its Digital
Measurement Solutions in the following way: It
enables brands to justify investments and prove
marketing success across new and emerging
channels. By creating a culture of measurement,
actions become informed by digital intelligence, not
by guesswork (Webtrends, 2013).
Used by hundreds of enterprises worldwide,
Webtrends provides a comprehensive, SharePoint-
specific solution that can help improve content,
usability, search and collaboration (Webtrends,
2013).
To paraphrase, this program helps to visualize the
data that drives your business and as a result identify
new areas for improvement in a scientific approach.
Of course it is also a basic tool for key figures which
have to be reported for quality management
requirements.
Within webtrends, a special profile for the
Clinical Knowledge Base has been created to
measure key metrics and analyze its benefit for the
users. Each page of the Clinical Knowledgebase
executes some JavaScript code on loading to collect
data of the current page, the referring page and the
user environment. In addition a "Tracking Pixel" is
requested from the central Webtrends Service. To
comply with the Siemens Privacy Guidelines and
Statements, all personal and private data is stripped
and the results are sent back to the central
Webtrends Service. The dashboard included for
example the following metrics (Fig.2).
The measurements are conducted on a monthly
basis. In addition, the number of user requests is
assessed and analyzed in the “Knowledge Activity
Report” (Fig.3).
4.2 Ongoing Evaluation of Research
Requests and Expert Advice
Services
As mentioned earlier, each user can ask for
individual medical expert advice or ask for a
WhenMeasuringPerformanceMakestheDifference-QualityandSuccessofaClinicalKnowledgeManagementProject
423
Figure 2: Dashboard Clinical Knowledge Base.
Figure 3: Knowledge Activity Report.
dedicated study. This can be done online in the
Clinical Knowledge Base via Infopath, a web form
application in sharepoint. The discussions are
documented along with scientific papers and final
expert advice. These requests are monitored based
on criteria like department, the requester's country of
origin, field of interest e.g. cardiology, oncology.
Finally there is the end-user satisfaction survey
in the middle of the year.
4.3 End-user Satisfaction Survey via
Sharepoint
The end-user / employee satisfaction survey is also
realized in the sharepoint environment. The online
employee satisfaction survey is a very effective way
to identify and diagnose the usefulness of the
information and services provided. Also, it provides
insight into upcoming new information needs.
The survey comprises overall dimension rating
questions as well as ratings by provided information
areas (Fig.4). Examples of overall rating questions
are “How satisfied have you been with the response
time?” and “How satisfied have you been with the
Figure 4: User Satisfaction Survey.
quality of information provided by the KM group?”.
The answers are rated on a five-point scale from
Excellent, Good, Average, Fair to Poor. Sample
questions concerning the content provided are “How
strongly do you need information from the topics
below in your daily work: innovations in medical
equipment? Clinical trends, statistics?”.
The free-text comment field is also very valuable
as the user can comment in a text box on what
information is missing or things to improve. Based
on this feedback, knowledge gaps are identified,
future knowledge demands are identified and the
service offerings can be further developed and
optimized. This process leads to the constant
KMIS2013-InternationalConferenceonKnowledgeManagementandInformationSharing
424
improvement of the available medical knowledge
and its dissemination.
5 CONCLUSIONS
KM as a whole is a complex process, which requires
more than an optimal information and
communication infrastructure. “The success of KM
initiatives depends equally on the active
involvement of everyone throughout the
organization, as well as on their consistency with the
organization’s broader business strategy and culture”
(Ergazakis, 2005).
Siemens Healthcare Clinical Knowledge
Management Project is continuously reviewed,
assessed and analyzed to identify the quality of its
knowledge assets and resources. At a more detailed
level, the company’s information system, its
processes and its knowledge enabling technology is
examined.
For many years, Siemens has followed a KM
with clear objectives and approaches and enjoyed
strong commitment from top management.
Since 2001, Siemens has been among the best
finalists in MAKE, the European Most Admired
Knowledge Enterprises ranking, which is conducted
annually by the UK-based consulting firm Teleos. In
2010, the organization achieved first place for the
third time (2003, 2004, 2010).
The Siemens Healthcare Academy, the Clinical
Competence Centers, as well as Clinical Knowledge
Base of Siemens Healthcare are a constructive
example of the company’s successful KM and
education strategy. The organizational benefit of the
Siemens Healthcare Clinical Knowledge
Management concept is the improved operational
efficiency of finding relevant information when
needed and greater confidence in the quality and
relevance of that information.
The whole point of KM is to make sure that the
knowledge available in an organization is applied
productively for the benefit of the organization.
REFERENCES
American Productivity and Quality Center (APQC), 2003.
Measuring the Impact of Knowledge Management. In
Best Practices Report.
Bhatti, W.A., Zaheer, A., Rehman, K.U., 2011. The effect
of knowledge management practices on organizational
performance: A conceptual study. African Journal of
Business Management, Vol.5 (7), pp. 2847- 2853.
Bose, R., 2003. Knowledge Management- enabled health
care management systems: capabilities, infrastructure,
and decision- support. Expert Systems with
Applications, Vol. 24, pp.59- 71.
Brown, G., Knowledge Management Metrics.
http://www.ispi.org/archives/resources/KnowledgeMa
nagementMetrics_Brown.pdf.
Davenport, T.H., De Long, D.W., Beers, M.C., 1998.
Successful Knowledge Management Projects. Sloan
Management Review, pp. 43-57.
DeLone, W.H., McLean, E.R., 2003/1992. The DeLone
and McLean model of information systems success: A
Ten- Year Update. Journal of Management
Information Systems, Vol.19, No.4, pp. 9-30.
Ergazakis, K. 2005 Knowledge Management in
Enterprises: a research agenda. Intelligent Systems in
Accounting, Finance and Management, Vol.13,pp. 17-
26.
Gamble, P.R., Blackwell, J. 2001, Knowledge
Management- A state of the art guide. British Library,
p. 203.
Hoss R., Schlussel, A., 2009. How do you measure the
Knowledge Management Maturity of your
Organization? Metrics that assess an Organization’s
KM State.
Jadoon, I.K., Hasnu, S.A.F., 2009. Collaboration
Dichotomies in Knowledge Management Success.
Journal of Knowledge Management Practice, Vol. 10
No. 4.
Jennex, M.E., Smolnik, S., Croasdell, D., 2008. Towards a
consensus knowledge management success definition.
In Strategies for Knowledge Management Success,
Information Science Reference.
Jennex, M.E., Smolnik, S., Croasdell, D., 2012. Where to
look for knowledge management success. IEEE,
pp.3969- 3978.
Metaxiotis, K., 2006. in Schwartz, D. G. Healthcare
Knowledge Management. Encyclopedia of Knowledge
Management, pp.204-210.
Robertson, J., 2003. Metrics for Knowledge Management
and Content Management. www.steptwo.com.au/
papers/kmc_metrics/index.html.
Webtrends, 2013. http://webtrends.com/products/analytics
and http://webtrends.com/products/integration/
sharepoint/.
WhenMeasuringPerformanceMakestheDifference-QualityandSuccessofaClinicalKnowledgeManagementProject
425
