Discussions of a Preliminary Hand Hygiene Compliance Monitoring

Application-as-a-service

Peng Zhang

, Marcelino Rodriguez-Cancio

, Douglas Schmidt

, Jules White

and Tom Dennis

Institute for Software Integrated Systems, Vanderbilt University, Nashville, TN, U.S.A.

Research Institute of Computer Science and Random Systems, University of Rennes 1, Rennes, France

ZH Solutions, Nashville, TN, U.S.A.

Keywords:

Machine Learning, Deep Learning, Data Mining, Hospital and Healthcare, Hand Hygiene Compliance,

Re-admissions.

Abstract:

Hospital Acquired Infections (HAIs) are a global concern as they impose signiﬁcant economic consequences

on the healthcare systems. In the U.S. alone, HAIs have cost hospitals an estimated $9.8 billion a year. An

effective measure to reduce the spread of HAIs is for Health Care Workers (HCWs) to comply with recom-

mended hand hygiene (HH) guidelines. Unfortunately, HH guideline compliance is currently poor, forcing

hospitals to implement controls. The current standard for monitoring compliance is overt direct observation

of hand sanitation of HCWs by trained observers, which can be time-consuming, costly, biased, and sporadic.

Our research describes a hand hygiene compliance monitoring app, Hygiene Police (HyPo), that can be de-

ployed as a service to alleviate the manual effort, reduce errors, and improve existing compliance monitoring

practice. HyPo exploits machine learning analyses of handwashing compliance data from a 30-bed intensive

care unit to predict future compliance characteristics. Based on the results, HyPo then provides HWCs with

timely feedback and augments the current monitoring approach to improve compliance.

1 INTRODUCTION

Emerging Concerns in Healthcare. Hospital Ac-

quired Infections (HAIs) are occupational infections

acquired by Healthcare Workers (HCWs) or by pa-

tients in healthcare facilities that appear after patient

discharge (WHO, 2009). HAIs represent signiﬁcant

health problems, with a considerable economic im-

pact on patients and hospitals worldwide.

Persistent exposures to diseases and lack of appro-

priate hand hygiene (HH) practice can cause HCWs’

hands to become carriers of infections transmitted to

patients or other staff through physical contact. To

reduce re-admission rates, therefore, HCWs are ex-

pected to comply with HH guidelines to prevent the

spread of HAIs in healthcare facilities. The cur-

rent standard practice of compliance monitoring is for

covert human auditors to unobtrusively observe and

record HH compliance of medical workers. Unfor-

tunately, this approach is costly and subject to bias

(Boyce et al., 2009) due to the Hawthorne effect (Eck-

manns et al., 2006), which occurs when subjects of a

study change their behavior due to awareness of being

observed.

Contribution. Based on our preliminary study on

HH compliance characteristics using machine learn-

ing (Zhang et al., 2016), this work proposes a hand

hygiene compliance monitoring app, Hygiene Police

(HyPo), that can be deployed as a service. The goal of

this app is to mitigate the laborious effort and reduce

errors of direct observation.

App Workﬂow. HyPo is implemented as a Java-

based desktop app that communicates to and from

Bluetooth Low-Energy (BLE) devices equipped at the

facility from our previous study (Zhang et al., 2016).

The schematic in Figure 1 depicts the overall app

workﬂow, which is divided into the following three

stages (the last two are the core components of HyPo):

1. Data Acquisition, where raw data is acquired

from the BLE devices.

2. Data Mining, where the raw data undergoes a

data mining process provisioned by HyPo to pro-

duce a set of features that is fed to Feature Selec-

tion algorithms to obtain a Sanitized Dataset. The

Feature Selection is done to improve the execu-

Zhang P., Rodriguez-Cancio M., Schmidt D., White J. and Dennis T.

Discussions of a Preliminary Hand Hygiene Compliance Monitoring Application-as-a-service.

DOI: 10.5220/0006293705370544

Figure 1: The Overall System Flow: from Data Collection to Post-prediction Analysis and Feedback Service.

tion performance of the Machine Learning (ML)

methods that will follow by determining the most

relevant features and removing the others from the

Sanitized Dataset.

3. Feedback Service, where the ML models are run

over the Sanitized Dataset to produce feature set

that can be used to provide timely feedback to

healthcare providers.

Paper Organization. The remainder of this paper

is organized as follows: Section 2 deﬁnes key terms

frequently referenced throughout the paper; Section 3

describes the data collection setup; Section 4 details

the data preparation and mining process; Section 5

describes Hypo’s feedback service and how it com-

plements the direct observation approach; Section 6

surveys and compares existing research in the area of

hand hygiene compliance improvement and monitor-

ing applications; and Section 7 presents concluding

remarks and outlines future extensions of this work.

2 HAND HYGIENE

COMPLIANCE OVERVIEW

This section deﬁnes the following terms that are used

frequently in the paper:

1. Hand hygiene opportunity: an opportunity for

hand cleaning is presented before each care

provider’s entry/exit of a patient room.

2. Hand hygiene/handwashing compliance: each

hand hygiene opportunity requires one hand hy-

giene action, which should be a correspond-

ing positive (compliance) or negative (non-

compliance) action (EFORE, 2009).

3. Entry compliance: hand hygiene compliance ob-

served at staff’s entry to a patient room, deter-

mined by wash on entry.

4. Exit compliance: hand hygiene compliance ob-

served at staff’s exit from a patient room, deter-

mined by wash on exit.

5. Wash on entry: hand hygiene action at patient

room entry that determines entry compliance, true

if performed and false otherwise.

6. Wash on exit: hand hygiene action at patient

room exit that determines exit compliance, true if

performed and false otherwise.

Our previous study collected 60 days of care

providers’ real-time location and handwashing data,

from an intensive care unit (ICU) equipped with 30

beds, and observed two major correlating factors to

compliance: (1) entry compliance has an 89% accu-

racy on predicting exit compliance and (2) exit com-

pliance can predict entry compliance at the next visit

(for the same staff) at an accuracy as high as 77%.

Likewise, location data was observed to have a minor

impact on predicting exit compliance (Zhang et al.,

2016).

Based on this previous study, in the HyPo app we

compiled the following rules of hand hygiene compli-

ance that ICU staff should abide by:

1. All on-duty staff at the ICU were required to wear

a BLE badge.

2. All staff were required to sanitize their hands

within a short interval of 2 minutes upon entering

a patient room and before exiting the same room.

3. Each compliant action should be associated with

an activation of a speciﬁc soap dispenser with

disinfectant solution against Clostridium difﬁcile,

a common HAI spread through physical con-

tact (Shrestha et al., 2016). These dispensers are

located both inside and outside each patient room.

These rules only apply to this ICU but can be con-

ﬁgured to work with other caregiving settings. The

rest of this paper describes the application of HyPo

using the same device-equipped 30-bed ICU from our

previous study (Zhang et al., 2016) as an example.

3 DATA ACQUISITION

This section describes the data acquisition process,

where real-time location data and handwashing sta-

tion activation data is recorded, and then provides an

overview of the essential data ﬁelds extracted from

the collection. The process described in this section

is one approach of obtaining the hand hygiene com-

pliance data to provide input for our app, but it is by

no means the only option to acquire this type of data.

3.1 Instrumentation Conﬁgurations

The ICU with HyPo deployment was equipped with a

Bluetooth Low-Energy (BLE) indoor positioning sys-

tem that provides room-level accuracy for reporting

staff locations in real-time. The system produced the

location data for all staff with BLE badges.

The ICU also deployed Gojo Active Monitoring

handwashing stations, which record each dispenser

activation. These activation events were then com-

bined with real-time location data to track individual

staff handwashing compliance. The system expected

to receive at least one handwashing event from ei-

ther a sanitation station inside of the room or a sta-

tion immediately outside the room within two min-

utes prior to entry, abiding the facility rules described

in Section 2. Similarly, two minutes before or af-

ter room exit, the system expected one handwashing

event from either sanitation stations.

Overall, the dataset collected at the studied ICU

contains 8 weeks of events recording activities from

180+ soap dispensers activated by 60 badged nurses

24 hours a day. All raw event data was streamed to a

data storage on Amazon Web Services (AWS), which

was post processed and output to a SQL database. We

then extrapolated the data ﬁelds of interest for com-

pliance predictions and analyses.

3.2 Dataset Limitations

Although real-time location data was acquired and

handwashing station activations recorded at the ICU,

the dataset was still an estimate rather than a ground

truth of hand hygiene compliance. The dataset col-

lected has a number of key threats to validity, includ-

ing: (1) not all staff wore their BLE badges at all

times, (2) the system could not differentiate activa-

tions from badged vs. non-badged visitors/staff, and

(3) subsets of the monitoring equipment went ofﬂine

at some intervals and prevented data capture in certain

rooms.

However, we did not consider these limitations

as fatal to our study results because we could either

easily eliminate the data entries associated with these

threats or discard the marginal impact that the threats

had on our ﬁndings.

3.3 Dataset Schema

From the SQL database we obtained an initial dataset

by omitting certain data ﬁelds with extraneous infor-

mation, such as device IDs of the wearable badges,

internally-used identiﬁers of the patient rooms, etc.

The data ﬁelds associated to each patient room visit

event that we deemed essential thus extracted from

the database include:

1. Staff ID - ID of badge worn by the staff who has

been associated with a patient room visit

2. Location - patient room number visited by the

badged staff

3. Entry time - timestamp (in CDT) at which the

badged nurse entered the patient room

4. Exit time - timestamp (in CDT) at which the

badged nurse exited the patient room

5. Wash on entry - a boolean value indicating

whether the staff properly performed hand hy-

giene on patient room entry

6. Wash on exit - a boolean value indicating if the

staff properly performed handwashing on patient

room exit

7. Duration - for how long (in milliseconds) the staff

was in the patient room

4 DATA PREPARATION

This section discusses how we prepared the collected

data to maximize the utilization of our machine learn-

ing classiﬁers, which is an important capability of-

fered by HyPo. This process is the same as that em-

ployed in (Zhang et al., 2016) to assist the analy-

ses and characterization of hand hygiene compliance.

Other inﬂuencing factors of hand hygiene compliance

may be discovered as more relevant data becomes

available, such as patient admittance details, medi-

cal records of admitted patients, facility regulations

of compliance, etc.

Despite the speciﬁcity of the dataset used through-

out this paper, the data mining process provided by

HyPo as described below can be generalized to sup-

port transformations of different forms of data col-

lected in other facilities.

4.1 Data Transformation

Most machine learning (ML) classiﬁers yield better

results when the input dataset is structured in certain

ways. For example, suppose we want to know if the

day of week (Monday to Sunday) inﬂuences compli-

ance, some ML classiﬁers will yield better results if

we express date as a set of integers ranging from 1

to 7, as opposed to a real continuous stream of times-

tamps expressed in milliseconds.

As another example, our location data consists of

room numbers, which provides little information re-

garding spatial distribution of the rooms. If we want

to know whether compliance decreases in nearby lo-

cations, we must ﬁrst transform the room numbers

into coordinates on the facility’s ﬂoor plan, for in-

stance.

To obtain a transformed schema that can be bet-

ter handled by our classiﬁers, we took the collected

dataset and performed the following transformations

over it:

1. We converted all event data from the original

timestamp format into an integer ﬁeld with range

1 to 7 to represent day of week, an integer ﬁeld

with range 1 to 4 to represent time of day in morn-

ing, afternoon, evening, bedtime, and another in-

teger data ﬁeld of 0-23 to represent hour of day.

The numeric representations of the original nom-

inal time stamp data will allow our classiﬁers to

achieve higher classiﬁcation accuracy.

2. We mapped each patient room on the ICU ﬂoor

plan to a set of x and y coordinates to identify the

spacial location. We then extended each entry in

the dataset to include these corresponding coordi-

nates of the patient room.

3. For each data point we added new ﬁelds to include

the previous record of the corresponding badged

staff’s handwashing data, i.e., duration, location,

washed on entry, and washed on exit. To ensure

data integrity, we removed all entries that did not

have previous handwashing records.

As a result of these transformations, we obtained

a new schema consisting of a minimal set of features

that our application expects to receive for best accu-

racy:

1. staff ID - integer

2. location (room number) - integer

3. washed on entry - TRUE/FALSE

4. washed on exit - TRUE/FALSE

5. duration (s) - length of patient room visit in sec-

onds, integer

6. entry hour - hour of day on room entry, 0-23

7. exit hour - hour of day on room exit, 0-23

8. entry time - time of day on recorded room entry

in Morning (1), Afternoon (2), Evening (3), and

Bedtime (4)

9. exit time - time of day on recorded patient room

exit, 1-4

10. entry day of week - day of week on recorded pa-

tient room entry, 1-7

11. exit day of week - day of week on room exit, 1-7

12. location X coordinate - x coordinate of patient

room on the ICU ﬂoor plan

13. location Y coordinate - y coordinate of patient

room on the ICU ﬂoor plan

14. previous duration (s) - duration of the same staff’s

previous patient room visit in seconds

15. previous washed on entry - dispenser activation on

previous room entry TRUE/FALSE

16. previous washed on exit - dispenser activation on

previous room exit TRUE/FALSE

17. previous location - previously visited patient room

number

4.2 Feature Selection

After we transformed the dataset into a features set,

we executed a feature selection process to automat-

ically select feature subsets in our transformed data

that best (1) reduced overﬁtting of data, (2) improved

classiﬁcation accuracy, and (3) decreased model train-

ing time (Guyon and Elisseeff, 2003). Although we

do not have a signiﬁcantly large feature list produced

for this ICU, it is still useful to apply this technique

to select the most relevant subsets of features to help

produce the most accurate feedback in the next step.

To automatically select features from the trans-

formed dataset, HyPo applies a supervised attribute

selection ﬁlter from the open source Weka Java li-

brary (Hall et al., 2009). The ﬁlter is composed of

two pieces: (1) a feature Evaluator to determine how

features are evaluated and (2) a Search Method to nav-

igate the feature’s search space. Our app runs feature

selection using the following pairs of Evaluators and

Search Methods, as shown in Table 1:

1. Evaluator: CfsSubsetEval that evaluates a subset

of features by considering each feature’s predic-

tive ability and the degree of redundancy between

them.

Search Method: GreedyStepwise with a backward

search through the space of attribute subsets.

2. Evaluator: InfoGainAttributeEval that evaluates

an attribute’s worth by measuring the information

gain with respect to the class variable to classify.

Search Method: Ranker that ranks features by

their individual evaluations with an optional pa-

rameter of 6 features in the output subset

3. Evaluator: WrapperSubsetEval (Kohavi and

John, 1997) with NaiveBayes (John and Langley,

1995) as the basic learning scheme and a 10-fold

cross validation to use for estimating accuracy.

Search Method: GeneticSearch that performs a

search using the simple genetic algorithm (Gold-

berg, 1989)

Table 1: Evaluator and Search Method Pairs Used in Fea-

ture Selection

Evaluator Search Method

CfsSubsetEval GreedyStepwise

InfoGainAttributeEval Ranker

WrapperSubsetEval GeneticSearch

Our previous study (Zhang et al., 2016) observed

two highly correlating factors of compliance using the

data collected in the same 30-bed ICU. We could con-

ﬁgure HyPo to select only these two features from

the dataset to use for determining feedback provi-

sion. However, automatic feature selection is an in-

tegral piece because as the dataset increases in size

and dimension, our enhanced app can continuously

combine features or intelligently adjust the correlat-

ing features to maximize classiﬁcation accuracy.

5 FEEDBACK SERVICE

This section ﬁrst describes the machine learning mod-

els employed by HyPo and then presents the feedback

service that uses these models to provide timely feed-

back and to complement the direct observation ap-

proach to hand hygiene compliance monitoring.

5.1 Machine Learning Models

After preparing the dataset, we split the data to 65%

for training, 10% for cross validation, and the re-

maining 25% for testing the ML models. Based

on the compliance prediction observations from the

previous study in (Zhang et al., 2016), we em-

ployed the top three classiﬁers, one from Weka (Hall

et al., 2009) and two deep nets from DeepLearning4J

(DL4J) (Team, 2016) to serve as our models for clas-

sifying washed on entry and washed on exit. HyPo

then uses the results with highest accuracy.

• The Sequential Minimal Optimization (SMO)

implementation of the Support Vector Machine

(SVM), which uses heuristics to partition the

training problem into smaller sub-problems and

uses pairwise linear regression to classify. This

method is usually resilient to data overﬁtting and

by default normalizes the input data (Platt et al.,

1998).

• The Feed-Forward Neural Network (FFNN),

which is a one direction (from input to output)

artiﬁcial neural network that performs classiﬁca-

tions based on weight calculations of the network

nodes (Glorot and Bengio, 2010). Using the DL4J

Java library, we developed a 3-layer FFNN with a

random seed of 6, 1000 iterations, a learning rate

of 0.1, and the Stochastic gradient descent opti-

mization algorithm (Gardner, 1984).

• The Recurrent Neural Network (RNN), which

has a feedback loop whereby the immediately pre-

vious step’s output is fed back to the net to affect

the outcome of the current step. We used a 3-layer

RNN with two Graves’ Long Short-Term Memory

(LSTM) layers (Graves, 2012) (input and hidden)

and an output layer along with the same parame-

ters as the FFNN.

5.2 Just-in-Time Alerting

With our previously characterized predictability of

compliance (Zhang et al., 2016), as described in Sec-

tion 2, and necessary pre-conﬁgurations to the data

collection instruments, HyPo can provide just-in-time

alerting to remind HCWs to perform hand hygiene

when they are predicted not to comply, using either a

singular prediction or a chain-prediction scheme, de-

pending on if there is adequate time to provide such

notiﬁcations between each hand hygiene opportunity.

Suppose that HyPo has just observed a staff

nurse’s compliance on a patient room entry, then the

ML classiﬁers will predict the same staff’s exit com-

pliance. For instance, if the staff is predicted to be

non-compliant, an alert of red ﬂashing light can be

sent to either the wearable badge or the device at the

appropriate dispenser activation station as a reminder

to the staff; otherwise, no alert is necessary.

If duration of the visit is too short of an interval

to send the notiﬁcation signal to the devices, then we

can use the probability chain rule (Schum, 1994) to

provide a backup alert to the same staff if necessary.

In this case, the ML models will use the predicted en-

try compliance for the current visit (from the staff’s

exit compliance of the previous visit) to determine

exit compliance of the current visit at a probability

of 89% * 77% = 69%. It is less ideal, but the likeli-

hood of the visit interval being too short is minimal

because the grace period for compliance is set at two

minutes, and if a room visit is within two minutes,

hand hygiene compliance is not required.

5.3 Recommend Training Material

If a staff member is frequently predicted as non-

compliant over a long observation period, HyPo (with

integrated email capabilities) can recommend hand

hygiene guidelines or appropriate training materials

to the staff via email. The goal is to improve compli-

ance on an individual basis.

5.4 Assist Direct Observation

The compliance prediction results can also be used to

assist the current standard practice of direct observa-

tion. With predicted non-compliance reoccurring at a

certain location (i.e., a patient room), HyPo can de-

ploy a human auditor (e.g., by sending a notiﬁcation)

to observe compliance at the location that should be

given most attention.

6 RELATED WORK

Due to worldwide high demands of HAI prevention, a

number of other researchers have studied approaches

to improve hand hygiene compliance. Although the

gold standard monitoring method is human-centric

(WHO, 2009), (Gould et al., 2011), a wide rage of

studies propose electronic or electronically assisted

hand hygiene compliance monitoring and interven-

tion systems (Ellingson et al., 2014), (Ward et al.,

2014). This section compares our work on the HyPo

app with common electronic intervention systems in-

cluding (1) technology-assisted direct human obser-

vation, (2) counting systems, and (3) automated mon-

itoring systems.

Technology-assisted Human Observation. Direct

observation is the most precise way of controlling

compliance. Several studies use technologies such as

handheld devices and cameras to aid human observa-

tion, aiming at reducing input errors, costs, and time

consumption. Handheld devices are used for data en-

try, and video cameras provide opportunities to reduce

the Hawthorne effect and observe locations that are

remote or hard to access.

Chen et al (Chen et al., 2013), used wireless data

entry devices and a website to allow human observers

to audit compliance. University of North Carolina

Hospitals implemented a “clean-in, clean-out” system

that allowed covert observers and designated nurses to

track compliance using a mobile app and a web portal

(Sickbert-Bennett et al., 2016).

Cameras have been used by Armellino (Armellino

et al., 2013) to increase compliance in an ICU. The

study connected motion sensors near the sinks that

would activate cameras being monitored by remote

auditors. The study by Davis (Davis, 2010) placed

a discreet camera at the entrance of a ward and as-

sessed compliance before and after a sink was placed

pointing to the dispenser.

Unfortunately, these methods still require human

interaction and can bias the results, as the medi-

cal workers know they are being directly observed.

Moreover, audits require trained personnel who are

regularly monitored to ensure quality control.

Counting Systems. Installing counting devices to

measure the remaining sanitation product volume or

the number of dispenser activation times is a quiet

method that is not subject to the Hawthorne effect.

A counter may detect usage patterns and frequency

changes.

Marras (Marra et al., 2010) used dispenser coun-

ters along with direct observation to assess whether

positive deviance in hand hygiene behaviors could

have an impact on reducing HAIs. A downside to

this approach, however, is that counter systems can-

not tell who used the dispensers and therefore are un-

able to evaluate compliance by itself. Morgan et al

(Morgan et al., 2012) provided evidence to support

the claim that dispenser usage data could be more re-

liable than direct human observation to estimate hand

hygiene compliance.

Automated Monitoring Systems using Wearables.

Many automated monitoring systems are capable of

producing feedback or reminders in real or near real

time without human intervention, similar to our ap-

proach.

Fakhry used a motion-triggered system with au-

dible hand washing reminders at each medical de-

partment entrance (Fakhry et al., 2012). Sahud and

Bhanot developed an electronic hand hygiene feed-

back device that reports real-time compliance rate on

a liquid-crystal display visible to all staff in the in-

tervention unit (Sahud and Bhanot, 2009). Edmond

et al installed a sensor network using a credit-card

sized sensor badge on each alcohol dispenser, which

when not activated on room entry or exit beeped with

a red indicating light (Edmond et al., 2010). Simi-

larly, Marra et al employed a wireless network with

sensors on the alcohol dispensers that provide real-

time ﬂashing light feedback to HCWs for hygiene ac-

tivity detection (Marra et al., 2014). Most recently,

Ellison et al proposed a prospective electronic hand

hygiene room entry/exit audible reminder system (El-

lison et al., 2015) that provides a combination of 24-

hour-a-day recording of hand hygiene activities and

real-time computer monitor performance feedback.

Differentiating Factors of Our Approach. All the

prior research we reviewed collected data to propose

strategies that increased hand hygiene performance or

gather conclusions regarding the efﬁcacy of a speciﬁc

approach. Our HyPo app is unique since it uses the

gathered data to predict future compliance behavior

instead of notifying appropriate caregivers after non-

compliance has been detected. Other approaches re-

act to non-compliance, while ours predicts compli-

ance ahead of time.

We presented a novel methodology using ML al-

gorithms, which is also unique to our work. Hence,

the aim of our work is also a differentiating factor.

In particular, HyPo evaluates the prediction capabili-

ties of different ML algorithms to predict compliance

ahead of time.

7 CONCLUDING REMARKS

This paper presented a hand hygiene monitoring app

called Hygiene Police (HyPo) that can be deployed as

a service to complement the current monitoring ap-

proach and improve compliance. We showed an ex-

ample data collection process taken place at a 30-bed

ICU where we acquired the handwashing compliance

data. We also described the data transformation pro-

cess HyPo employs to maximize the utilization of the

selected machine learning (ML) classiﬁers.

Combining the results of real-time compliance

predictions using the correlations identiﬁed in (Zhang

et al., 2016), HyPo can provide three types of ser-

vices: (1) just-in-time alerting to remind predicted

non-compliant staff to perform hand hygiene, (2)

recommending training materials to habitually non-

compliant staff via email, and (3) assisting the direct

observation approach by deploying human auditors at

the opportune time and place when and where non-

compliance is frequently predicted to occur. We also

compared our app to related research work and found

that our approach predicted future compliance behav-

ior instead of reacted to non-compliance as in other

approaches. Our methodology using ML algorithms

is unique and is the only work that evaluates ML pre-

diction capabilities in this domain.

In future work, we plan on collecting more com-

pliance data, ideally using the same process as dis-

cussed in the paper. We will use this data to ﬁne tune

the parameters in our ML classiﬁers to increase the

prediction accuracy. We will also run simulations that

test whether our HyPo app can improve compliance in

general and if the improvement can be sustained over

time in a range of caregiving settings.

REFERENCES

Armellino, D., Trivedi, M., Law, I., Singh, N., Schilling,

M. E., Hussain, E., and Farber, B. (2013). Replicating

changes in hand hygiene in a surgical intensive care

unit with remote video auditing and feedback. Ameri-

can Journal of Infection Control, 41(10):925–927.

Boyce, J. M., Cooper, T., and Dolan, M. J. (2009). Evalua-

tion of an Electronic Device for Real-Time Measure-

ment of Alcohol-Based Hand Rub Use. Infection Con-

trol & Hospital Epidemiology, 30(11):1090–1095.

Chen, L. F., Carriker, C., Staheli, R., Isaacs, P., Elliott,

B., Miller, B. A., Anderson, D. J., Moehring, R. W.,

Vereen, S., Bringhurst, J., Rhodes, L., Strittholt, N.,

and Sexton, D. J. (2013). Observing and improving

hand hygiene compliance: Implementation and reﬁne-

ment of an electronic-assisted direct-observer hand

hygiene audit program. Infection control and hospi-

tal epidemiology: the ofﬁcial journal of the Society of

Hospital Epidemiologists of America, 34(2):207–210.

Davis, C. R. (2010). Infection-free surgery: how to improve

hand-hygiene compliance and eradicate methicillin-

resistant Staphylococcus aureus from surgical wards.

Annals of the Royal College of Surgeons of England,

92(4):316–319.

Eckmanns, T., Bessert, J., Behnke, M., Gastmeier, P., and

uden, H. (2006). Compliance with antiseptic hand

rub use in intensive care units the hawthorne effect.

Infection Control, 27(09):931–934.

Edmond, M. B., Goodell, A., Zuelzer, W., Sanogo, K.,

Elam, K., and Bearman, G. (2010). Successful use of

alcohol sensor technology to monitor and report hand

hygiene compliance. The Journal of Hospital Infec-

tion, 76(4):364–365.

EFORE, B. (2009). Hand hygiene technical reference man-

ual.

Ellingson, K., Haas, J. P., Aiello, A. E., Kusek, L., Mara-

gakis, L. L., Olmsted, R. N., Perencevich, E., Pol-

green, P. M., Schweizer, M. L., Trexler, P., VanAm-

ringe, M., and Yokoe, D. S. (2014). Strategies to Pre-

vent Healthcare-Associated Infections through Hand

Hygiene. Infection Control and Hospital Epidemiol-

ogy, 35(8):937–960.

Ellison, R. T., Barysauskas, C. M., Rundensteiner, E. A.,

Wang, D., and Barton, B. (2015). A Prospective

Controlled Trial of an Electronic Hand Hygiene Re-

minder System. Open Forum Infectious Diseases,

page ofv121.

Fakhry, M., Hanna, G. B., Anderson, O., Holmes, A., and

Nathwani, D. (2012). Effectiveness of an audible re-

minder on hand hygiene adherence. American Journal

of Infection Control, 40(4):320–323.

Gardner, W. A. (1984). Learning characteristics of

stochastic-gradient-descent algorithms: A general

study, analysis, and critique. Signal Processing,

6(2):113–133.

Glorot, X. and Bengio, Y. (2010). Understanding the dif-

ﬁculty of training deep feedforward neural networks.

In Aistats, volume 9, pages 249–256.

Goldberg, D. E. (1989). Genetic algorithms in search, opti-

mization and machine learning. Addison-Wesley.

Gould, D. J., Drey, N. S., and Creedon, S. (2011). Rou-

tine hand hygiene audit by direct observation: has

nemesis arrived? The Journal of Hospital Infection,

77(4):290–293.

Graves, A. (2012). Supervised sequence labelling. In Super-

vised Sequence Labelling with Recurrent Neural Net-

works, pages 5–13. Springer.

Guyon, I. and Elisseeff, A. (2003). An introduction to vari-

able and feature selection. Journal of machine learn-

ing research, 3(Mar):1157–1182.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann,

P., and Witten, I. H. (2009). The weka data min-

ing software: an update. ACM SIGKDD explorations

newsletter, 11(1):10–18.

John, G. H. and Langley, P. (1995). Estimating continuous

distributions in bayesian classiﬁers. In Eleventh Con-

ference on Uncertainty in Artiﬁcial Intelligence, pages

338–345, San Mateo. Morgan Kaufmann.

Kohavi, R. and John, G. H. (1997). Wrappers for feature

subset selection. Artiﬁcial Intelligence, 97(1-2):273–

324. Special issue on relevance.

Marra, A. R., Guastelli, L. R., Arajo, C. M. P. d., Santos,

J. L. S. d., Lamblet, L. C. R., Silva, M., Lima, G. d.,

Cal, R. G. R., Paes, n. T., Neto, M. C., Barbosa, L.,

Edmond, M. B., and Santos, O. F. P. d. (2010). Pos-

itive Deviance A New Strategy for Improving Hand

Hygiene Compliance. Infection Control & Hos-

pital Epidemiology, 31(1):12–20.

Marra, A. R., Sampaio Camargo, T. Z., Magnus, T. P.,

Blaya, R. P., Dos Santos, G. B., Guastelli, L. R.,

Rodrigues, R. D., Prado, M., Victor, E. d. S., Bo-

gossian, H., Monte, J. C. M., dos Santos, O. F. P.,

Oyama, C. K., and Edmond, M. B. (2014). The use

of real-time feedback via wireless technology to im-

prove hand hygiene compliance. American Journal of

Infection Control, 42(6):608–611.

Morgan, D. J., Pineles, L., Shardell, M., Young, A., Elling-

son, K., Jernigan, J. A., Day, H. R., Thom, K. A., Har-

ris, A. D., and Perencevich, E. N. (2012). Automated

hand hygiene count devices may better measure com-

pliance than human observation. American Journal of

Infection Control, 40(10):955–959.

Platt, J. et al. (1998). Sequential minimal optimization: A

fast algorithm for training support vector machines.

Sahud, A. G. and Bhanot, N. (2009). Measuring hand hy-

giene compliance: a new frontier for improving hand

hygiene. Infection Control and Hospital Epidemiol-

ogy, 30(11):1132.

Schum, D. A. (1994). The evidential foundations of proba-

bilistic reasoning. Northwestern University Press.

Shrestha, S. K., Sunkesula, V. C., Kundrapu, S., Tomas,

M. E., Nerandzic, M. M., and Donskey, C. J. (2016).

Acquisition of clostridium difﬁcile on hands of health-

care personnel caring for patients with resolved c. dif-

ﬁcile infection. Infection Control & Hospital Epi-

demiology, 37(04):475–477.

Sickbert-Bennett, E. E., DiBiase, L. M., Schade Willis,

T. M., Wolak, E. S., Weber, D. J., and Rutala, W. A.

(2016). Reducing health careassociated infections by

implementing a novel all hands on deck approach for

hand hygiene compliance. American Journal of Infec-

tion Control, 44(5, Supplement):e13–e16.

Team, D. D. (2016). Deeplearning4j: Open-source dis-

tributed deep learning for the jvm. Apache Software

Foundation License, 2.

Ward, M. A., Schweizer, M. L., Polgreen, P. M., Gupta,

K., Reisinger, H. S., and Perencevich, E. N. (2014).

Automated and electronically assisted hand hygiene

monitoring systems: A systematic review. American

Journal of Infection Control, 42(5):472–478.

WHO (2009). A guide to the implementation of the WHO

multimodal hand hygiene improvement strategy.

Zhang, P., White, J., Schmidt, D., and Dennis, T. (2016). A

preliminary study of hand hygiene compliance char-

acteristics with machine learning methods. (ISIS-16-

101).