Collaborative Filtering for Identifying Prescription Omissions in an ICU

Anima Singh and John Guttag

Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, U.S.A.

Keywords:

Machine Learning, Collaborative Filtering, Latent Factor Models, Intensive Care Units, Prescription Omis-

sions.

Abstract:

Medication errors in critical care are frequent and can lead to adverse consequences. One important cate-

gory of errors is prescription omission, i.e., failure to prescribe a potentially useful medication. Studies have

shown that failure to prescribe a medication can result in adverse consequences leading to patient morbidity or

even patient mortality (Aspden et al., 2007; Olsen et al., 2007). In this paper, we present a machine learning

based approach to building a system that can be used to provide physicians with an ordered list of possible

omissions. We investigated three different collaborative ﬁltering approaches as well as simple prevalence and

co-occurrence methods. When evaluated on over 19,000 ICU admissions, each of the CF approaches outper-

formed both prevalence and co-occurrence based methods. This work highlights the importance of capturing

a multi-scale view of the prescription data for the task of identifying omissions. Our results suggest that latent

factor models and neighborhood models are better at capturing different kinds of omissions. Latent factor

models demonstrated improved performance on identifying omission of rarely prescribed medications while

neighborhood models were slightly better at identifying omissions of commonly prescribed medications.

1 INTRODUCTION

Medication-related errors account for 78% of the

medical errors in the intensive care unit (ICU) (Roth-

schild et al., 2005). Patients in critical care are partic-

ularly vulnerable to such errors. First, medication er-

rors are more frequent in an ICU because patients are

given twice as many medications as patients outside

of a critical care setting (Cullen et al., 1997). Second,

patients in an ICU are high-risk patients, and therefore

any error is more likely to have serious consequences.

The inpatient medication use-process can be

broadly categorized into ﬁve phases: prescription,

transcription, preparation, dispensation and adminis-

tration. A medication error can occur in any one of

the stages in the process. According to the Institute

of Medicine (IOM), the prescription stage is most fre-

quently associated with errors (Aspden et al., 2007).

One important category of errors is prescription omis-

sion, i.e., failure to prescribe a potentially useful med-

ication. Studies have shown that failure to prescribe a

medication can result in adverse consequences lead-

ing to patient morbidity or even patient mortality (As-

pden et al., 2007; Olsen et al., 2007). In addition,

these errors translate into longer hospital stays. In this

paper, we focus our discussion on ways to detect pos-

sible prescription omissions in an ICU.

Medication safety can be improved by incorpo-

rating automated systems that can alert caretakers

about likely medication errors. Because of the in-

creasing availability of electronic medical records, the

prospect of developing automated systems that can

use patient information to help guide prescription is

promising. Electronic medical records contain infor-

mation about a patient’s demographics, medications,

diagnosis and other medical information. While it is

challenging for a caretaker to process all the informa-

tion, computerized systems can process such informa-

tion, and provide useful insights in detecting errors. In

particular, these systems can provide signiﬁcant ben-

eﬁts in reducing prescription omission errors by pro-

viding suggestions about likely omissions.

Although prescription omissions are a serious

problem, most admissions do not have omissions.

Given a list of medications a patient is currently on, an

ideal system should alert a caretaker if there is a high

likelihood of an omission. In addition, it should offer

suggestions about which medication might be miss-

ing.

In this paper, we investigate development of auto-

mated systems that can learn to perform the task of

identifying possible prescription omissions. One of

Singh A. and Guttag J..

Collaborative Filtering for Identifying Prescription Omissions in an ICU.

DOI: 10.5220/0004233900580064

In Proceedings of the International Conference on Health Informatics (HEALTHINF-2013), pages 58-64

ISBN: 978-989-8565-37-2

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

the challenges in doing this is that there is no reli-

able data set containing ground truth. There are many

data sets that contain information about which drugs

are prescribed, but as far as we know there are no

signiﬁcant data sets that contain information about

which drugs should have been prescribed but were

not. Therefore, rather than trying to directly learn

typical errors, we try to learn which drugs are typi-

cally prescribed together and how prescriptions relate

to diagnoses. The system might learn, for example,

that patients with a diagnosis of hypertension who are

prescribed Atenolol (a beta blocker) are usually also

prescribed Chlorthalidone (a diuretic). We then use

the absence of a ‘typical’ association as an indication

that a drug may have been omitted.

The high variation in the ways in which drugs are

used, combined with the relatively low incidence of

omissions makes it implausible under most circum-

stance to predict with high certainty that a drug is

missing. Therefore we choose to provide a ranked list

of the most probable omissions. A physician could

look at the ﬁrst few items on this list as candidate

drugs to think about prescribing rather than as an in-

dication that he or she has likely made an error.

We use collaborative ﬁltering (CF) as the basis

of our approach to learning. This seems a promis-

ing choice for at least two reasons: 1) CF has been

widely used in recommender systems that infer infor-

mation about a user’s interest in items based on past

behavior and preferences, and then provide rankings

for other items. 2) CF methods are robust at handling

sparse datasets, i.e., datasets in which most of the en-

tries are zeros or missing. This is important in our

context, since even the sickest patients receive only a

tiny subset of the possible drugs during an admission.

Hasan et al. have explored neighborhood based

and logistic regression based CF to detect omissions

in an outpatient setting (Hasan et al., 2011). Our work

focuses on capturing a multi-scale view of the data. In

addition to a neighborhood approach, we investigate

latent factor models that aim to learn prescription pat-

terns from data. We also explore models that combine

both methods.

We evaluated our approach on data from MIMIC

II (Saeed et al., 2002), a clinical database of ICU pa-

tients. During the learning phase, we assumed that all

records accurately reﬂected the drugs that were pre-

scribed.

Since the database does not include an indication

of whether or not a drug was omitted, we simulated

omissions by deleting drugs that had been prescribed.

Ideally, one would use a deletion model that mirrored

the kinds of mistakes actually made in practice. For

example, if physicians more often omit rarely pre-

scribed drugs, our deletion model should contain that

bias. Unfortunately, there does not seem to be any

publicly available information about the actual pattern

of omission errors. This led us to use a simple omis-

sion model in which we remove one of the prescribed

medications uniformly at random.

Using this model, we evaluate different types of

CF approaches, and compare them to the performance

of prior and co-occurrence based methods. We use

each method to rank each medication that was not pre-

scribed based on the likelihood that it is omitted. We

then evaluate each method based on the rank of the

actually omitted medication in the list. The lower the

rank, the better the approach at identifying omissions.

When evaluated on over 19,000 ICU admissions,

each of the CF approaches outperformed both prior

and co-occurrence based methods. Overall, a latent

factor CF approach that captures prescription patterns

performed better than a nearest neighbor approach.

The latent factor models outperformed the nearest

neighbor approach at identifying omitted drugs that

are prescribed less often. On the other hand, neigh-

borhood models performed better at identifying omis-

sions of commonly prescribed drugs. This suggests

that combining the two approaches can potentially

improve performance.

The remainder of the paper is organized as fol-

lows. Section 2 discusses problem formulation and

notations used in the rest of the paper. Section 3 de-

scribes different methods that we investigated. Sec-

tion 4 summarizes the data used for this study. Our

experimental methodology and results are presented

in Section 5. Finally, Section 6 presents the summary

and future work.

2 PROBLEM FORMULATION

In this section, we present the formal representation of

our problem and introduce notations. We consider the

ﬁrst 24 hours of a hospital admission during which

a patient was admitted to an ICU. We represent an

admission as a list of prescribed medications and the

diagnoses of the patient at the time of admission. In

the rest of the paper, we will refer to medications and

diagnoses as as items.

Let,

• {a

, a

, ..., a

} be a set of vectors that represent

ICU admissions in the data

• {m

, m

, ..., m

} be the set of medications

• {d

, d

, ..., d

} be the set of diagnoses

• I = M + D be the number of items

CollaborativeFilteringforIdentifyingPrescriptionOmissionsinanICU

We represent the j

admission a

as a binary vector

of length I where,

For i ≤ M,

(

1, if medication m

is prescribed.

0, otherwise.

(1)

For i > M,

(

1, if the patient has a diagnosis d

i−M

0, otherwise.

(2)

The information about all admissions is repre-

sented as a I × A matrix R, where R

i j

∈ {0, 1} (Fig-

ure 1).

1" 0" 0" 0" 0" 0" 1" 0" 1" 0" 0" 0" 0" 0" 1"

0" 0" 0" 0" 1" 1" 0" 1" 0" 0" 1" 0" 0" 0" 0"

1" 0" 0" 1" 0" 0" 1" 0" 0" 1" 0" 1" 0" 0" 0"

0" 0" 1" 0" 0" 0" 0" 0" 0" 0" 0" 0" 0" 1" 0"

0" 0" 0" 0" 1" 0" 1" 0" 0" 0" 0" 0" 1" 0" 1"

0" 0" 0" 0" 0" 0" 0" 0" 1" 0" 1" 0" 0" 0" 0"

0" 0" 0" 1" 0" 1" 0" 1" 0" 0" 0" 0" 0" 0" 0"

0" 0" 0" 0" 0" 0" 1" 0" 0" 0" 0" 0" 0" 0" 0"

0" 0" 0" 0" 0" 0" 0" 0" 0" 0" 0" 1" 0" 1" 1"

0" 0" 1" 0" 0" 1" 0" 0" 0" 0" 1" 0" 1" 0" 0"

0" 0" 0" 0" 0" 0" 0" 1" 0" 0" 0" 0" 0" 0" 0"

0" 0" 0" 1" 1" 0" 0" 0" 1" 0" 0" 0" 0" 0" 0"

0" 1" 0" 0" 0" 0" 1" 0" 0" 1" 0" 0" 0" 1" 1"

0" 0" 0" 0" 0" 1" 0" 0" 0" 0" 0" 0" 0" 0" 0"

0" 0" 0" 0" 0" 0" 0" 1" 0" 0" 0" 0" 1" 0" 0"

0" 0" 1" 1" 1" 0" 0" 0" 0" 1" 0" 1" 0" 0" 0"

0" 0" 0" 0" 0" 0" 1" 0" 0" 0" 0" 0" 0" 0" 1"

0" 1" 0" 0" 0" 0" 0" 0" 1" 0" 1" 0" 0" 0" 0"

0" 0" 0" 1" 0" 1" 0" 1" 0" 0" 0" 1" 0" 0" 0"

0" 0" 0" 0" 0" 0" 0" 0" 0" 1" 0" 0" 0" 0" 1"

ICU"admissions"

" a

..."

Medica5ons"

Diagnosis"

..."

Figure 1: An I × A binary matrix R where I = M + D.

3 METHODS

In this section, we discuss the methods we investi-

gated for identifying prescription omission.

For an admission a

, each method considers all

medications m

for which R

i j

= 0. It then assigns

a score to each medication. Based on the score, the

method generates a ranked list of possible omissions

as the output.

3.1 Prior based Approach

The prior based approach assigns a score that is equal

to the prior of the medication based on ICU admis-

sions in the training set T. The prior of medication

, P

, is deﬁned as the fraction of ICU admissions

in T that are prescribed m

score(i, ·) = P

(3)

A prior based approach will perform well when

medications that are prescribed frequently are most

likely to be omitted. We use this approach as a base-

line to evaluate performance of other models that we

investigate.

3.2 Co-occurrence based Approach

Our co-occurrence based approach assigns a score for

medication m

based on how many times it co-occurs

with other medications that are observed in admission

. The co-occurrence is calculated based on admis-

sions in the training set T.

score(i, j) =

∑

d∈{z|R

z, j

=1,z≤M

}

∑

∈T

i,k

d,k

(4)

This approach is the simplest way to capture some

information about which sets of medications are pre-

scribed together. However, it only captures pair-wise

associations of prescribed medications.

3.3 Collaborative Filtering

Collaborative ﬁltering has been used extensively in

recommendation systems to infer user’s preferences

from historical data. Analogous to the recommen-

dation system application, where there are users and

items, our CF formulation uses ICU admissions (as

users) and information about the ICU admission (as

items).

There are two main categories of CF techniques:

neighborhood approaches and latent factor model ap-

proaches.

3.3.1 Neighborhood Approach

To make a recommendation for a user u, a neighbor-

hood approach starts by ﬁnding users that are similar

to u. Next, it takes weighted combinations of their

ratings to make recommendations for u. In our work,

we treat ICU admissions as users, and consider two

ICU admissions to be similar if they share many med-

ications and diagnoses.

The most commonly used neighborhood based-

approach is kNN. Given an ICU admission a

, we ﬁnd

k other ICU admissions that are most similar with re-

spect to a weighted cosine similarity-based distance

metric (Xie et al., 2012).

The standard cosine similarity metric calculates

distance between two vectors where each component

of the vector is weighted equally. However, equal

weighting is not appropriate for our application. For

example, two ICU admissions that share a less com-

monly prescribed drug such as Carbamazepine (used

to control seizures) should probably be considered

more similar than two admissions that both prescribed

Acetaminophen. The weighted cosine similarity be-

tween two hospital admissions a

and a

is deﬁned as:

∑

i=1

∑

i=1

)

∑

i=1

)

(5)

HEALTHINF2013-InternationalConferenceonHealthInformatics

where,

• for i ≤ M, w

= 1-P

• for i > M, w

= 1- P

i−M

The predicted score for medication m

in admis-

sion a

is obtained by taking a weighted average of the

values of m

in the k nearest neighbors while adjusting

for the prior of the medication. The term (R

i,t

− P

)

can be interpreted as information over the prior.

score(i, j) = P

∑

∈N

i,t

− P

)

∑

∈N

(6)

where, N is the set of k nearest neighbors.

This formulation of nearest neighbors is good at

capturing localized relationships provided that the

distance between the k nearest neighbors and the

given ICU admission is small.

3.3.2 Latent Factor Models

Latent factor models aim to uncover latent features

that explain the observed data. These models have

been successfully used in Netﬂix movie recommenda-

tions (Li et al., 2010). In our application, we attempt

to uncover combinations of medications and diagno-

sis that appear frequently in the dataset. We refer to

these as prescription patterns.

Latent factor models are generated using matrix

factorization methods. The main idea of matrix fac-

torization is to approximate matrix R using two lower

rank matrices X (an I × F matrix) and Y (an F × A

matrix) such that

i j

≈ (XY )

i j

∑

f =1

i f

f j

(7)

The F columns of matrix X are the basis vec-

tors. Each basis vector encodes information about

prescription patterns learned from the dataset. Each

column of matrix Y contains weights for each basis

vector. An ICU admission a

is approximated as a

linear combination of the basis vectors weighted us-

ing the j

column of Y .

Again, to adjust for the prior, instead of approxi-

mating the matrix R, we approximate the residual ma-

trix R.

i, j

= R

i, j

− P (8)

where,

P =

(

, i ≤ M.

i−M

, i > M.

(9)

We learn matrices X and Y by solving the fol-

lowing regularized optimization problem using an al-

ternating least squares algorithm (Zamir and Gabriel,

1979):

min

X,Y

∑

i, j

i j

− X

i·

· j

)

+ λ(kX

i·

+ kY

· j

) (10)

where X

i·

is the i

row of matrix X and Y

· j

is the j

column of matrix Y .

Once we estimate matrices X and Y , the predicted

score for medication m

in admission a

is obtained

by:

score(i, j) = P

+ X

i·

· j

(11)

While neighborhood approach rely on the nearest

neighbors to predict omissions, latent factor models

have a more holistic approach. They attempt to un-

cover prescription patterns in the data. A latent factor

model with F factors captures the F most prominent

basis vectors, which can be viewed as encoding pre-

scription patterns, from the data.

3.3.3 Integrated Approach

In the integrated approach, we combine the latent fac-

tor model with the neighborhood model. This inte-

grated model is aimed at capturing both prescription

patterns and localized relationships in the data. Here

we model the score as having three terms:

score(i, j) = P

+ X

i·

· j

+ H(i, j) (12)

The ﬁrst two terms are identical to that in Equa-

tion 11. The third term attempts to capture any resid-

ual information that the baseline and the latent factor

model are unable to account for.

After computing matrices X and Y , we compute

H(i, j) using the neighborhood approach.

H(i, j) =

∑

∈N

i,t

− P

− X

i·

· j

)

∑

∈N

(13)

4 THE DATA

We obtained our data from the MIMIC II (Multi-

parameter Intelligent Monitoring in Intensive Care)

database. MIMIC II contains comprehensive clinical

data of ICU patients in Beth Israel Deaconess Medical

Center in Boston, Massachusetts.

MIMIC II contains 24,097 admissions with 928

unique generic medication names. For our study, we

removed medications that were prescribed in < 0.1%

of the admissions. Those medications were removed

CollaborativeFilteringforIdentifyingPrescriptionOmissionsinanICU

Table 1: The most commonly prescribed medications in our

data.

Medication N (%)

Sodium chloride electrolyte 9721 (50.2%)

Acetaminophen 9031 (46.6%)

Heparin 8875 (45.8%)

Pantaprazole 8309 (42.9%)

Aspirin 6511 (33.6%)

because it would be difﬁcult to learn any useful pat-

terns from few examples using a data driven ap-

proach. This left 316 unique medication names. We

also removed admissions in which only one medica-

tion was prescribed. This resulted in 19,394 admis-

sions.

The most commonly prescribed medications in the

hospital admissions are listed in Table 1.

For each admission, we include information about

the 30 diagnoses that form the Elixhauser comorbidity

system. The Elixhauser comorbidity system is com-

monly used in health research. It has been found to

be signiﬁcantly associated with in-hospital and post-

discharge mortality. It uses 30 binary variables that

represent 30 diagnoses. The most common diagnoses

in our data are listed in Table 2.

Table 2: The most common diagnoses in our data.

Diagnoses N (%)

Hypertension 5959 (30.8%)

Fluid and electrolyte disorders 4233 (21.9%)

Congestive heart failure 3714 (19.2%)

Cardiac arrythmias 3609 (18.6%)

Diabetes (uncomplicated) 3466 (17.9%))

5 EXPERIMENTS AND RESULTS

This section describes the set of experiments used to

compare different methods and presents a discussion

of the results.

5.1 Experimental Setup

We divided our data into training and test sets. Our

training set was composed of 12,994 randomly se-

lected ICU admissions, while our test set consisted of

the remaining 6400 admissions. We used the train-

ing set for training purposes and for parameter se-

lection. Parameter selection was carried out using 5-

fold cross-validation. During parameter selection, we

tested k in the range [10, 100, 500, 1000, 2000] for the

neighborhood approach. For the latent factor mod-

els, we evaluated F in the range [100, 150, 200, 300]

and λ in the range [20, 30, 50, 70, 100]. The test set

was used to evaluate the different methods described

in Section 3.

As discussed earlier, MIMIC II does not contain

annotations that indicate if there was a prescription

omission. Therefore, we used a simulation model to

generate them. Since we were unable to ﬁnd litera-

ture with a good model for actual admission errors,

we chose a simple model. For each admission a

the test set, we randomly remove one medication m

that is prescribed.

Our goal is to identify the medication that was

omitted during the simulation. For every test admis-

sion, each method generates a ranked list of medica-

tions that are not prescribed to the patient during the

admission. The rankings are based on the scoring sys-

tems presented in Section 3.

5.2 Results and Discussion

Table 3 compares the overall performance of the dif-

ferent methods. The comparison is based on the me-

dian ranking of the omitted drug over all of the admis-

sions in the test set. A lower rank for the omitted drug

means that it appeared near the top of the list and thus

corresponds to better performance.

The co-occurrence approach (Co-occur) that cap-

tures pairwise association of medications signiﬁ-

cantly outperformed the baseline method (Prior).

All three CF approaches, the neighborhood approach

(Prior+kNN), the latent factor model (Prior+MF)

and the integrated model (Prior+MF+kNN) signiﬁ-

cantly outperformed both Prior and Co-occur.

Table 3: Median rank of the omitted medication using dif-

ferent methods on the test set. A lower rank means better

performance.

Method Parameters used Median Rank

Prior - 17

Co-occur - 8

Prior+kNN k=500 5

Prior+MF F= 100; λ=20 4

Prior+ MF+kNN F= 100; λ=20; k=1000 4

Overall, Prior+MF yielded a small improvement

in performance over Prior+kNN. However, if one

considers only ICU admissions in which the omit-

ted drug has a low prior, the performance improve-

ment is considerably more. Figure 3 shows the per-

formance of Prior+MF relative to Prior+kNN as a

function of the prior of the omitted medication. A

positive difference in rank means that Prior+MF out-

performed Prior+kNN. For admissions with omitted

medications with a low prior, Prior+MF strongly out-

performs Prior+kNN. For admissions in which the

HEALTHINF2013-InternationalConferenceonHealthInformatics

0 5 10 15 20 25 30 35 40 45 50

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Top x medications

(Average) Fraction of holdout admissions for which omitted drug is identified

(a) Performance on crossvalidation sets

Prior

Cooccur

Prior+kNN

Prior+MF

Prior+MF+kNN

0 5 10 15 20 25 30 35 40 45 50

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Top x medications

(Average) Fraction of test admissions for which omitted drug is identified

(b) Performance on the test set

Prior

Cooccur

Prior+kNN

Prior+MF

Prior+MF+kNN

Figure 2: Cumulative performance curves. Each data point shows the fraction of the admissions for which the omitted drug

was identiﬁed in the top x medications. (a) Cumulative performance curves obtained from 5 fold cross validation. The error

bars represent one standard deviation of uncertainty. (b) Cumulative performance curves on the test set.

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

ï1

Prior of the omitted medication

Difference in rank

Figure 3: The median difference in rank when Prior+MF

outperformed and underperformed Prior+kNN in the test

set. Each bin corresponds to a quartile of the prior of the

omitted medications.

omitted medication had a high prior, Prior+kNN did

slightly better. And for admissions where the omit-

ted medication had a middling prior, the performance

of the two methods was almost identical. Keep in

mind that our method of simulating drug omissions

results in more high prior drugs than low prior drugs

being omitted. Thus, when evaluating overall perfor-

mance, Prior+MF’s strong advantage on a few omis-

sions of low prior drugs is largely balanced out by

Prior+kNN’s small advantage on the omissions of

more common drugs.

We believe that the association of performance

with prevalence stems from the fact that omitted drugs

with high priors are well represented in the k nearest

neighbors. Therefore, Prior+kNN is able to achieve

a better rank for the omitted drug. On the other

hand, when the omitted drug has a lower prior, the

neighbors are less informative and prescription pat-

terns learned using latent factor models fare better at

identifying such omissions.

Figure 2 shows the cumulative performance for

each method on both cross validation (CV) and test

sets for the top 50 medications. The cumulative

performance is measured using the mean fraction of

test admissions for which the omitted drug was found

in the top x medications. For example, Figure 2(a)

shows that for 31.2% of the CV test admissions,

Prior included the omitted drug within the top 5

medications. Prior + kNN and Prior + MF found

the omitted drug (within the top 5 medications)

for 55.2% and 58.1% of the CV test admissions

respectively. The error bars in Figure 2(a) show one

standard deviation of uncertainty.

Consistent with our previous observation,

Prior+MF outperforms Prior+kNN. Notice that the

error bars for Prior+kNN and Prior+MF do not

overlap for the top 15 medications. Of course, all of

the curves must start to converge as we consider an

ever larger number of drugs, since when x = M the

curves meet at 1.0.

The results discussed thus far suggest that the

latent factor models and the neighborhood approach

are better at identifying different kinds of omissions.

The proposed integrated model (Prior+MF+kNN)

combines both methods, and is aimed at capturing

multi-scale information - both local relationships and

higher level prescription patterns, that exist in the

data. Disappointingly, the overall performance of

our integrated model was not signiﬁcantly different

from that of the latent factor model in our dataset.

However, the cumulative performance curves in

Figure 2 show that Prior+MF+kNN consistently

outperforms both Prior+MF and Prior+kNN in both

CV and test sets. These suggest that a different

approach to an integrated model is probably worth

CollaborativeFilteringforIdentifyingPrescriptionOmissionsinanICU

pursuing.

Figure 2(b) shows the cumulative performance on

the test set. The trends in the cumulative performance

curves are similar. This shows the generalizability of

the methods.

6 SUMMARY & FUTURE WORK

In this paper, we investigated a collaborative ﬁlter-

ing framework for identifying prescription omission

in ICU admissions. We investigated three different

types of CF methods: a nearest neighbor approach, a

latent factor model and an integrated model. Using

prescription medication data and diagnosis informa-

tion from 19,000 hospital admissions, we tackled the

problem of identifying which medications are likely

to be omitted. The novelty of our work lies in using

CF to investigate multi-scale information, i.e., local-

ized relationships as well as overall prescription pat-

terns, contained in prescription data for the task of

identifying prescription omissions.

All of the CF approaches ranked omitted medica-

tion higher than a prior based algorithm and a simple

co-occurrence approach. Overall, latent factor models

outperformed neighborhood models. Further analy-

sis showed that latent factor models are much better

at identifying omissions of rarely prescribed medica-

tions. Neighborhood models, on the other hand, were

able to yield a slightly better ranking for commonly

prescribed medications. This highlights the relevance

of learning both strong local relationships and overall

patterns from the data for the application of identify-

ing omissions.

For our methods, we envision a use model where

a computerized systems will suggest a list of medica-

tions that are likely to be omitted given the informa-

tion about the patient. A physician then scans through

the top x medications to look for potential omissions.

We conclude with a brief discussion of some fu-

ture work. First, our current integrated approach

optimizes latent factor models independently of the

neighborhood approach. Since our results indicate

that the neighborhood approach and latent factor

model perform better for different kinds of omissions,

an improved approach that intelligently uses localized

relationships and/or prescription patterns will likely

result in better performance. We plan to investigate

advanced models that optimizes latent factor param-

eters, taking into account the information about the

neighbors.

Our work explores CF methods using medication

and diagnosis data for identifying omissions. Clearly,

incorporating other information in electronic medical

records, e.g., a patient’s allergies should yield im-

proved results. Furthermore, we investigated only

data-driven approaches. Hybrid computerized sys-

tems that augment data driven methods with expert

knowledge are also worth pursuing.

ACKNOWLEDGEMENTS

The authors gratefully acknowledge the ﬁnancial as-

sistance provided by Quanta Computers Inc.

REFERENCES

Aspden, P., Wolcott, J., Bootman, J. L., and Cronenwett,

L. R. (2007). Preventing Medication Errors: Quality

Chasm Series. The National Academies Press.

Cullen, D., Sweitzer, B., Bates, D., Burdick, E., Edmond-

son, A., and Leape., L. (1997). Preventable adverse

drug events in hospitalized patients: a comparative

study of intensive care and general care units. Crit-

ical Care Medicine, 25(8):1289–1297.

Hasan, S., Duncan, G. T., Neill, D. B., and Padman, R.

(2011). Automatic detection of omissions in medi-

cation lists. Journal of the American Medical Infor-

matics Association : JAMIA, 18(4):449–58.

Li, Y., Hu, J., Zhai, C., and Chen, Y. (2010). Improving

one-class collaborative ﬁltering by incorporating rich

user information. In Proceedings of the 19th ACM in-

ternational conference on Information and knowledge

management, CIKM ’10, pages 959–968, New York,

NY, USA. ACM.

Olsen, S., Neale, G., Schwab, K., Psaila, B., Patel, T., Chap-

man, E. J., and Vincent, C. (2007). Hospital staff

should use more than one method to detect adverse

events and potential adverse events: incident report-

ing, pharmacist surveillance and local real-time record

review may all have a place. Quality & safety in health

care, 16(1):40–4.

Rothschild, J., Landrigan, C., Cronin, J., Kaushal, R., Lock-

ley, S., Burdick, E., Stone, P., Lilly, C., Katz, J.,

Czeisler, C., and Bates, D. (2005). The critical care

safety study: The incidence and nature of adverse

events and serious medical errors in intensive care.

Critical Care Medicine, 33(8):1694–1700.

Saeed, M., Lieu, C., Raber, G., and Mark, R. (2002). Mimic

ii: A massive temporal icu patient database to support

research in intelligent patient monitoring. In Comput-

ers in Cardiology, 2002, pages 641 – 644.

Xie, J., Leishman, S., Tian, L., Lisuk, D., Koo, S., and

Blume, M. (2012). Feature Engineering in User‘s Mu-

sic Preference Prediction. Journal of Machine Learn-

ing Research: Workshop and Conference Proceed-

ings, 18:183–197.

Zamir, S. and Gabriel, K. R. (1979). Lower Rank Approxi-

mation of Matrices by Least Squares With Any Choice

of Weights. Technometrics, 21(4).

HEALTHINF2013-InternationalConferenceonHealthInformatics