AN INVESTIGATION OF THE USE OF KMS

ON CLINICAL PERFORMANCE

Maryam Alavi

, Steven L. Johnson

and Youngjin Yoo

Goizueta Business School, Emory University, 1300 Clifton Road, Atlanta, GA, U.S.A.

Fox School of Business and Management, Temple University, Philadelphia, PA, U.S.A.

Keywords: Knowledge Management Systems (KMS) Applications and Use, Organizational Impacts of KMS, KMS in

Medical Settings.

Abstract: Use of Information Technology (IT) to facilitate storage, search and access to codified knowledge can

enhance organizational performance. Considering the knowledge intensive nature of medical decision

making, use of knowledge management systems (KMS) can potentially enhance delivery of patient care.

This paper reports on a field study that investigated the impact of use of a KMS by emergency physicians on

their decision making behaviors (admission of patients for in-hospital care) and efficiency of care. The

findings indicate that in the emergency room (ER) under study, the use of a KMS for placing diagnostic

orders resulted in lower hospital admissions, cost savings, and a shorter ER stay.

1 INTRODUCTION

An extensive body of research based on knowledge-

based view of organizations (for example, Alavi and

Leidner, 2001; Spender, 1996; Nonaka and

Takeuchi, 1995) suggests that Information

Technology (IT) can enhance organizational

performance through consistent and effective

application of knowledge in organizational practices.

Considering the knowledge-intensive nature of the

healthcare domain, the high rate of change and

innovation, and the wide scope of knowledge needed

by the professionals in this field, patient care can

greatly benefit from advanced information

technologies that facilitate storage, search, and

timely access to the best knowledge available.

In the U.S., IT is viewed as integral to achieving

substantial quality and efficiency improvements in

healthcare delivery and management. A key idea

underlying this view is the use of IT to support

knowledge management (KM) to enhance and

facilitate clinical decision-making. The goal is to

apply best available evidence gained from scientific

method to delivery of patient care. It seeks the ideal

in which all medical professionals should make

explicit, conscious and judicious use of all available

best knowledge in making clinical decisions.

Proponents argue that application of evidence-based

practice would reduce the inconsistencies and

uncertainties in patient care. Many institutions try to

implement evidence-based clinical decision-making

in the form of policy, recommendations, guidelines

and regulations. However, translating such

institutional behaviors into actual behaviors

practiced by healthcare professionals is a

challenging task. IT is seen by many as an integral

solution to close the gap in implementing evidence-

based clinical decision-making practices. By

codifying best available knowledge and integrating

it into the electronic medical record (EMR) systems

that healthcare professionals interact with to manage

various clinical information, hospitals and medical

practices might be able to implement the evidence-

based practice more consistently and improve their

performance.

The literature, however, is clear that mere

stocking of knowledge through codification is not

enough to improve organizational performance

(Alavi and Tiwana, 2002). Knowledge needs to be

applied in practice in order to produce intended

outcomes (Cook and Brown, 1999; Orlikowski,

2002). Thus, the potential benefits KMS cannot be

realized through the codification and accumulation

of medical knowledge in IT systems alone. Rather, it

is in the ability to take effective action by applying

knowledge. Thus, it is not clear to what extent KMS

will be effective for Physicians who make complex

236

Alavi M., L. Johnson S. and Yoo Y..

AN INVESTIGATION OF THE USE OF KMS ON CLINICAL PERFORMANCE.

DOI: 10.5220/0003669902360240

In Proceedings of the International Conference on Knowledge Management and Information Sharing (KMIS-2011), pages 236-240

ISBN: 978-989-8425-81-2

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

clinical decisions in constrained timeframes and

under high degree of uncertainty.

In order to address this issue, we have conducted

a field study to examine the impact of KMS in an

emergency room (ER) at a major hospital in the

United States. Specifically; we investigated the

following research questions:

Research Question 1: How does the use

knowledge management system change the clinical

decision-making behaviors by medical

professionals?

Research Question 2: What are the impacts of the

use of knowledge management system on cost and

efficiency in clinical decision-making?

2 A FIELD STUDY OF KMS USE

IN AN EMERGENCY ROOM

Lately, the popularity and deployment of EMR

(electronic medical records) have been on the rise in

the U.S. EMR systems are integrated IT systems for

healthcare information management and process

support. We anticipate that the eventual ubiquitous

availability of EMR systems in the U.S. will create

the necessary “backbone” that serves as the

infrastructure for knowledge codification, storage,

search, and delivery for clinical decision-making.

This in turn creates a need for conducting studies

that provide the necessary insights and

understanding for development and deployment of

knowledge management systems (KMS) for

effective and efficient delivery of patient care.

The effectiveness and outcomes of the decisions

made in an emergency room depend on timely and

accurate diagnosis and delivery of appropriate

treatments to patients. These decisions are in turn

impacted by the timely availability of the requisite

knowledge such as the knowledge of diagnostic tests

and their outcomes, treatment protocols, and

accurate presentation of the patient’s condition and

symptoms. As such, clinical decisions in emergency

rooms can benefit from the codification,

accumulation, and delivery of knowledge to

augment physicians’ judgment and know-how.

We investigated the impact of the use of a KMS

by emergency room physicians on the rate of

admission of emergency room patients to the

hospital, emergency room charges, and patient time

spent in the ER. The emergency room under

investigation serves the 573-bed university hospital

and is staffed by 38 physicians, working between

one to fifteen 8-hour shifts per month. The ER

treated approximately 93 patients per day and in

2009, a total of about 34,000 patients were treated at

this facility.

The emergency room in our field study routinely

collects and maintains patient records in a data

warehouse. The records of all adult patients (18

years and older) who visited the ER in a 321 day

period between January 2009 and November 2009

were used in this study (the precise dates were

masked to protect patient anonymity). For the

purpose of this study, we analyzed the records of ER

patients who complained of abdominal pain as their

primary symptom. We chose to focus on abdominal

pain complaints because they represent relatively

ambiguous cases and can potentially benefit the

most from the use of a KMS. Considering patient

privacy issues, we used a de-identified data sample.

This was accomplished by creating new data sets

from the warehouse patient records by excluding all

identifying fields, assigning appropriate aliases, and

copying the remaining data fields. The data fields

used in our analyses are described in Table 1.

The IT infrastructure in the ER consisted of an

EMR system, FirstNet, by Cerner Corporation in

Kansas City, Missouri. FirstNet functionalities

include electronic records and notes, results

management (e.g., lab and radiology reports),

clinical provider order entry (e.g., test orders), and a

KMS for decision support (e.g., standard diagnostic

orders and clinical guidelines and protocols). The

KMS module provides physicians with

recommended diagnostic tests and medication order

sets, based on the patient’s symptoms. As such, this

module can enhance the accuracy of diagnosis and

delivery of appropriate patient care. The standard

order sets (the codified knowledge embedded in the

system) are developed by expert physicians based on

the best available evidence of their efficacy.

Physicians and nurses, however, can choose not to

use the KMS recommendation and place their own

orders.

We focused our investigation on the impact of

the use of diagnostic order sets placed through the

KMS on patient care outcomes.

In order to control for the exogenous impact of the

severity of the conditions that affect the outcome of

the ER visit, we only focused on two most prevalent

acuity levels (urgent and emergent), resulting in a

sample of 2238 emergency department visits. A

small number of immediate, stable, and non-urgent

cases were dropped. Table 2 shows basic descriptive

statistics and the correlations between variables. All

correlations with an absolute value of 0.07 or greater

are significant at the p <0.05 levels.

AN INVESTIGATION OF THE USE OF KMS ON CLINICAL PERFORMANCE

237

Table 1: Emergency Room (ER) Patient Record Fields

Included in the Study.

Field Description

Admitted to

Hospital

Outcome of ER stay resulted in

admission to hospital for in-

patient care

Duration of ER Stay Length of stay in ER

Total Charges

Total of doctor charges and

hospital charges for this ER stay

(available only for a subset of

patients)

Acuity

Level of acuity of patient

condition as assessed at time of

arrival to ER

Age

Age of patient on date of

admission

Gender Gender of patient

First Diagnostic

Order through KMS

During this ER stay the first

diagnostic order was placed via

the KMS (a standard care set)

Total Number of

Diagnostic Orders

Total number of diagnostic orders

placed during this ER stay

Number of KMS

Diagnostic Orders

Of the total number of diagnostic

orders placed during this ER stay,

the number placed through KMS

Rate of orders

placed through

KMS

Number of diagnostic orders

through KMS divided by ER stay

duration

Average Diagnostic

Order Placement as

% of ER Duration

Considering all diagnostic orders

placed during this ER stay, the

average time of order placement

as a percentage of stay duration

3 PRELIMINARY RESULTS

We ran separate analyses to answer the research

questions. In the first analysis, we examined if the

use of recommendations from KMS affect hospital

admissions. In particular, we focused on the impact

of the use of the system (represented by Ordering

Rate through KMS) on specific clinical decision-

making behaviors (represented by Total Number of

Diagnostic Orders, Average Diagnostic Order, and

First Diagnostic Order through KMS). Since hospital

admission is a binary variable, we used the logistic

regression. Table 3 presents the logistic regression

results for the predictors of hospital admissions.

Model 1 contains three significant control variables

related to the inherent complexity of patient care:

acuity, age, and gender. The parameter estimates for

acuity and age have the expected signs and

magnitude: higher acuity patients are far more likely

(~ 3x as likely) to be admitted to the hospital than

lower acuity patients; each additional year of patient

age leads to a very slight (~ 1.02x) increase in

hospital admission. Model 1 has an AIC value of

2739 and a log-likelihood of -1365 (df=4). Model 2

is a full test with our explanatory variables. It has an

AIC value of 2294 and a log-likelihood of -1138

(df=9). This is a statistically significant

improvement over model 1 in model fit (p<0.001).

Our results show that the use of KMS in clinical

decision making contributes to the reduction of

hospital admission. Furthermore, if the first

diagnostic order follows the recommendation from

the system, the reduction of the admission rate is

even larger.

In the second analysis, we explored if the use of

recommendations from KMS affected patient care

cost. Since the total charge does not follow the

normal distribution, we used a log transformation.

The results of an OLS regression model of

predictors of total charges (log transformed) are

presented in Table 4. This model includes only the

788 emergency room encounters for which both

physician charges and emergency department

hospital charges are available. For those patients

who were eventually admitted to the hospital,

hospital charges were not available as the charges

were rolled into other hospital charges incurred

during the in-patient care. Because a portion of total

charges are directly associated with the duration of a

patient’s emergency room stay, duration is included

as an additional control variable. As expected, our

results show that the duration of stay and total

number of orders are the main drivers of the total

charges. Our results also show that if the physician

starts with a diagnostic order recommended by the

KMS, it leads to a significant reduction in total

charges.

Given that duration is one of the key variables

that affect the total cost, in the third analysis, we

examined if the use of recommendations from the

KMS can contribute to the reduction of the duration.

The results of an OLS regression model of

predictors of duration of stay in the emergency

department are presented in Table 5.

Taken together, our analyses show several

statistically significant relationships for emergency

room encounters with patients. The variables of

acuity, age, and total number of diagnostic orders all

increase the likelihood of an encounter resulting in a

hospital admission. Also, female patients are less

likely to result in hospital admission. However, we

found that when a physician places a higher rate of

diagnostic orders through the KMS, it significantly

reduces the chance that the patient gets admitted in

the hospital. Furthermore, this relationship is

strengthened when the first diagnostic order is

placed through the KMS.

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

238

Table 2: Correlations (n-2238).

mean s.d. 1 2 3 4 5 6 7 8 9

Admitted to Hospital

(1=Yes)

0.40 0.49

2 Duration of ER Stay 0.38 0.23 0.43

3 Acuity (1=Higher) 0.34 0.47 0.29 0.07

4 Age 44.6 18.2 0.23 0.09 0.16

5 Gender (1=Female) 0.67 0.47 -0.13 0.02 -0.17 -0.09

First Diag. Order through

KMS (1=Yes)

0.69 0.46 0.05 0.03 0.04 0.03 -0.01

Total Number of Diagnostic

Orders

11.0 4.6 0.45 0.29 0.20 0.20 -0.02 0.23

Number of KMS Diagnostic

Orders

4.00 4.56 0.08 0.07 0.02 0.09 -0.04 0.59 0.20

KMS Diagnostic Ordering

Rate

13.1 14.3 -0.12 -0.31 -0.02 0.01 -0.02 0.47 0.02 0.76

Average Diagnostic Order

Placement as % of ER

Duration

0.22 0.14 0.08 -0.13 -0.04 -0.00 0.03 -0.15 0.19 -0.09 -0.06

Table 3: Logistic Regression for (In-Patient) Hospital Admission (n=2238).

Model 1: Controls Model 2: Full Model

Est.

Exp

(Est.)

Std.

Error

Sig. Est.

Exp

(Est.)

Std.

Err.

Sig.

(Intercept) -1.58 0.21 0.18 *** -3.72 0.02 0.25 ***

Acuity 1.10 3.02 0.10 *** 0.91 2.49 0.11 ***

Age 0.02 1.02 0.00 *** 0.02 1.02 0.00 ***

Gender -0.37 0.69 0.10 *** -0.55 0.58 0.11 ***

First Diagnostic Order through KMS -0.12 0.89 0.15

Total Number of Diagnostic Orders 0.26 1.30 0.02 ***

KMS Diagnostic Ordering Rate -0.08 0.92 0.02 ***

Average Diagnostic Order Placement as % of

ER Duration

0.23 1.26 0.41

Interaction: First Order through KMS

KMS Diagnostic Ordering Rate

0.05 1.06 0.02 **

** p < 0.01; *** p < 0.001

In the subset of patient encounters where

hospital charge data are available, patient age, the

duration of emergency department stay, and the

total number of diagnostic orders are all associated

with higher charges. Female patients and

encounters where the diagnostic orders are placed

earlier in the stay are all associated with lower

charges. If the physician starts the diagnostic

orders following the recommendations from the

system, it leads to a significant reduction of total

cost.

In the full data set the measures associated with

a longer duration of emergency department stay

were gender (female) and the total number of

diagnostic orders. Encounters with higher acuity

and encounters where the diagnostic orders were

placed earlier in the stay had lower durations.

Again, if the physician starts the diagnostic orders

following the recommendations from the system, it

leads to a significant reduction in the duration of

patient stay in ER.

AN INVESTIGATION OF THE USE OF KMS ON CLINICAL PERFORMANCE

239

Table 4: OLS Regression for Total Charges (Log

Transformed) (n=788).

Estimate

Std.

Error

Sig.

(Intercept) 6.18 0.10 ***

Acuity 0.04 0.05

Age 0.00 0.00 **

Gender -0.09 0.05 *

Duration 2.02 0.19 ***

First Diagnostic Order

through KMS

-0.22 0.07 ***

Total Number of

Diagnostic Orders

0.12 0.01 ***

KMS Diagnostic

Ordering Rate

0.00 0.00

Average Diagnostic

Order Placement as % of

ER Duration

-0.87 0.17 ***

Interaction: First Order

through KMS * KMS

Diagnostic Ordering Rate

0.00 0.00

*p < 0.05; ** p < 0.01; *** p < 0.001; Adj. R

= 0.47

Table 5: OLS Regression for ER Duration (n=2238).

Estimate

Std.

Error

Sig.

(Intercept) 0.30 0.02 ***

Acuity -0.03 0.01 ***

Age 0.00 0.00

Gender 0.04 0.01 ***

Admitted to Hospital 0.19 0.01 ***

First Diagnostic Order

through KMS

-0.03 0.01 **

Total Number of

Diagnostic Orders

0.01 0.00 ***

Average Diagnostic

Order Placement as %

of ER Duration

-0.36 0.03 ***

** p < 0.01; *** p < 0.001; Adj. R

= 0.24

4 CONCLUSIONS

An open question—and one that Information

Systems (IS) researchers are well-situated to help

address—is if and when knowledge management

systems may actually impact practice. It is merely

not enough to identify best practices; it is a greater

challenge to consistently enact them. Our study

provides preliminary evidence in how the

implementation of KMS for diagnostic testing in

an ER is leading to positive patient outcomes.

REFERENCES

Alavi, M. and Leidner, D. (2001). Knowledge

Management and Knowledge Management Systems:

Conceptual Foundations and Research Issues, MIS

Quarterly, 25, 107-136.

Alavi, M. and Tiwana, A. (2002). Knowledge

Integration in Virtual Teams: The Potential Role of

KMS, Journal of the American Society for

Information Science and Technology, 53, 1029-

1037.

Cook, S. D. N. and Brown, J. S. (1999). Bridging

Epistemologies: The Generative Dance between

Organizational Knowledge and Organizational

Knowing, Organization Science, 10, 381-400.

Nonaka, I. and Takeuchi, H. (1995). The Knowledge

Creating Company: How Japanese Companies

Create the Dynamic Innovation. New York: Oxford

University Press.

Orlikowski, W. J. (2002). Knowing in Practice: Enacting

a Collective Capability in Distributed Organizing,

Organization Science, 13, 249-273.

Spender, J. C. (1996). Making Knowledge the Basis of a

Dynamic Theory of the Firm, Strategic Management

Journal, 17, 45-62.

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

240