Augmenting Guideline-based CDSS with Experts’ Knowledge

Nekane Larburu

1,2

, Naiara Muro

1,2

, Iván Macía

1,2

, Eider Sánchez

, Hui Wang

, John Winder

Jacques Boaud

6,7

and Brigitte Séroussi

7,8

eHeatlh and Biomedical Applications, Vicomtech-IK4, Donostia-San Sebastian, Spain

Biodonostia, Donostia-San Sebastian, Spain

NARU, Donostia-San Sebastian, Spain

Computer Science Research Institute, Ulster University, Newtownabbey, U.K.

Institute of Nursing and Health Research, Ulster University, Newtownabbey, U.K.

AP-HP, DRCD, Paris, France

Sorbonne Universités, UPMC Univ. Paris 06, INSERM, Université Paris 13, Sorbonne Paris Cité,

UMR S 1142, LIMICS, Paris, France

AP-HP, Hôpital Tenon, DSP, Paris, France

Keywords: Evidence Based Medicine, Breast Cancer, Computer Interpretable Clinical Guidelines, CDSS.

Abstract: Over the past years, clinical guidelines have increasingly become part of the clinical daily practice in order to

provide best available Evidence-Based-Medicine services. Hence, their formalization as computer

interpretable guidelines (CIG) and their implementation in clinical decision support systems (CDSSs) are

emerging to support clinicians in their decision making process and potentially improve medical outcomes.

However, guideline compliancy in the clinical daily practice is still “low”. Some of the reasons for such low

compliance rate are (i) lack of a complete guideline to cover special clinical cases (e.g. oncogeriatric cases),

(ii) absence of parameters that current guidelines do not consider (e.g. lifestyle) and (iii) absence of up-to-

date guidelines due to lengthy validation procedures. In this paper we present a novel method to build a CDSS

that, besides integrating CIGs, stores experts’ knowledge to enrich the CDSS and provide best support to

clinicians. The knowledge includes new evidence collected over time by the systematic usage of CDSSs.

1 INTRODUCTION

In order to offer the best available care, medical

practice adopts the Evidence-Based-Medicine (EBM)

principle, defined as “the conscientious, explicit and

judicious use of current best evidence in making

decisions about care of individual patients” (Sackett

et al., 1996). In the 90s clinical practice guidelines

(CPGs) start to appear as rigorous evaluations of

different clinical activities that improved the clinical

practice and developed health care processes

(Grimshaw and Russell, 1994), so that clinicians

could follow EBM. However, clinicians still found

barriers to adhere to CPGs (Cabana MD et al., 1999).

Some of these barriers were lack of awareness, lack

of familiarity, lack of agreement, lack of outcome

expectancy or the inertia to previous practice. These

barriers are still valid in the current practice.

In order to overcome some of the main obstacles,

during the last decade multiple CPGs have been

formalized in an electronic way, i.e. computer

interpretable guidelines (CIG), and applied in Clinical

Decision Support Systems (CDSSs) (B. Séroussi et

al., 2013). Nevertheless, it was discovered that CPGs

still have limitations. For instance, in the context of

breast cancer (BC) some factors such as elderly

patients, multifocal tumours, occurrence of

micrometastasis on lymph-node and patient choice

are causes of CPGs non-compliance (Chéreau et al.,

2011; Landercasper et al., 2006; Lebeau et al., 2011;

B. Séroussi et al., 2013).

In this paper we present a method to acquire

expert knowledge in order to develop a knowledge-

augmented guideline-based CDSS. It results in the

development of new tools to support clinicians on

their decision making process for cases that have low

evidence (e.g. oncogeriatric cases) or where other

aspects (e.g. patient preferences) are crucial.

370

Larburu N., Muro N., MacÃ a I., SÃ ˛anchez E., Wang H., Winder J., Boaud J. and SÃl’roussi B.

Augmenting Guideline-based CDSS with Expertsâ

Z Knowledge.

DOI: 10.5220/0006213903700376

In Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2017), pages 370-376

ISBN: 978-989-758-213-4

The rest of this paper is organized as follows.

Section 2 presents the state of the art on CIG

compliance. Section 3 presents the method to

augment the guideline-based CDSS with expert´s

knowledge. Section 4 presents the application of such

method in an EU project, DESIREE, developed in the

context of breast cancer. Section 5 proposed a short

discussion on the presented method and Section 6

concludes the paper and gives some future work lines.

2 STATE OF THE ART

2.1 Guideline Compliance

Variations in medical practices have been observed

for decades, questioning the quality of care (Mercuri

and Gafni, 2011). CPGs compliance is one of the

primary performance measures to assess the quality

of medical practice. McGlynn et al. (McGlynn et al.,

2003) reported that 54.9% of the studied patient

population received CPGs’ based recommended care,

which vary from 10% to 78%. In their work they

reported that for BC 75.7% were consistent with

recommended care, based on 9 quality indicators.

Other studies also demonstrated suboptimal guideline

compliance levels in BC (Adegboyega et al., 2015;

Landercasper et al., 2006; Lebeau et al., 2011;

Wöckel et al., 2010). The published levels of

guideline compliance range from 12% (Lebeau et al.,

2011) to 100% (Adegboyega et al., 2015), depending

on the definition of guideline compliance and the

level of abstraction of the guideline. For instance,

Wöckel et al. (Wöckel et al., 2010) reported 80% of

adherence to German-S3-BC guideline for surgery

and for hormone therapy, and 71% for chemotherapy,

indicating different compliance levels for the

different components of the care plan. Similarly,

Lebeau et al. (Lebeau et al., 2011) reported high level

of guideline compliance, but also said that

“management of non-metastatic BC was fully

compliant (considering jointly 20 quality criteria)”.

2.2 Causes Associated with Guideline

Non-Compliance

The causes of variations in care delivery are

multifactorial. A review by Flottorp et al. (Flottorp et

al., 2013) identified a list of 51 determinants of

practice grouped in seven domains: guideline factors,

individual health professional factors, patient factors,

professional interactions, incentives and resources,

capacity for organisational change, and social,

political, and legal factors.

However, effective guideline-based CDSSs

(Beeler et al., 2014; Roshanov et al., 2013) provide a

framework for logging non-compliance cases and

learn from them. As demonstrated by Séroussi et al.

(B. Séroussi et al., 2013), guideline compliance

increases by using guideline-based CDSS.

Additionally, Bouaud et al. (Bouaud and Séroussi,

2011) determined the main factors related with CPG

non-compliance and reported the distribution of non-

compliance causes. Here, we list these causes

reported in (Bouaud and Séroussi, 2011):

 Patient preferences: When patients receive

more complete information about the benefits

and risks of different treatment options, the

patients made their own active, informed

decisions (Leonard et al., 2011). This decision

is influenced by their personal preferences.

 Evolution of medical knowledge: CPG

knowledge may not consider latest scientific

publications and clinical essays, and hence,

they may lag behind ‘last’ evidence (B.

Séroussi et al., 2013). This may include that

‘new’ parameters are not being considered in

the applied guidelines.

 Specific situations: Rare situations that require

specific clinical research are also a cause of

non-compliance. For example, in BC scenarios

shown in (Parks et al., 2012; Schnitt, 1998; B

Séroussi et al., 2013), microinvasion,

neadjuvant situations and oncogeriatry

conditions are the main causes that lead into

non-compliance situations.

 Medical choices: One of the main cause of

non-compliance is a medical decision that is

prioritized over the guideline recommendation.

For example, in (Bouaud and Séroussi, 2011),

the study reported that BC multidisciplinary

staff meetings’ choice (i.e. breast units choice)

is the main reason reported as the cause for

CPG non-compliance.

 Others: Finally, it may be other reasons that

lead into CPG non-compliancy that do not

belong to any of the previously reported causes.

Some studies provide tools to support clinicians

in understanding the reasons of non-compliancy

(Hussain et al., 2007). Others exploit the stored

patient information to predict patient worsening and

prevent potential emergencies (Colantonio et al.,

2008). Yet, there is no evidence that all the

information related to the whole decision making

process (such as additional patient data, the decision

criteria for giving a specific treatment and patient

Augmenting Guideline-based CDSS with Expertsâ

Z Knowledge

371

outcomes) is stored and exploited over time to enrich

the CDSS and provide better decision support to

decision makers in prospective cases.

3 METHOD TO AUGMENT

GUIDELINE-BASED CDSS

Here we present a method that enables the

exploitation of the implicit knowledge used in a

decision making process. The method is presented in

the following subsections: Section 3.1 presents the

starting point, which applies the clinical guideline

model, Section 3.2 presents the second stage, which

describes the acquisition process of experts’

knowledge and Section 3.3 presents how such

experts’ knowledge is exploited.

3.1 Clinical Guideline Model

As discussed in Section 1, CPGs are intended to

optimize patient care. Therefore, in this initial stage a

clinical guideline model is developed. The clinical

guideline model incorporates (i) different guidelines

based on users’ needs, (ii) updated clinical guidelines

or studies, so that the provided recommendations

correspond to the latest available evidence, and

detects (iii) potential inconsistencies that could be

reflected on the implemented guidelines.

3.2 Experts’ Knowledge Acquisition

The second stage of this method focuses on experts’

knowledge acquisition and storage.

We developed a flexible solution that enables the

storage of each decisional event (Figure 1). Each

decisional event reflects all the rationality for taking

a decision and the consequences of such decision.

Hence, we define the decisional event as 









,



, 



,,, where (i) P

is a set of

patient parameters involved in the decision-making

process, (ii) R

is a set of clinical conditions (e.g.

rules) wherein such parameters have been analysed,

which results in a set of recommendations (iii) FD is

a final decision that is taken by the decision maker,

(iv) C

is a set of criteria for which the final decision

is made (which could be a patient parameter), (v) E is

the executed treatment (usually, same as the final

decision) and (vi) O(t) is the health outcomes of a

patient measured over time t (e.g. (“ICHOM –

International Consortium for Health Outcomes

Measurement,” n.d.)).

Figure 1: Decisional event and decisional history.

As shown in Figure 1, the storage of decisional

events over time lead into a decisional history. The

decisional history is later used to retrieve conclusions

or discover new knowledge (Section 3.3).

3.3 Experts Knowledge Exploitation

Here we present the three usages of this decisional

history: (i) recommendations assessment, (ii) patient

similarity based recommendations and (iii)

knowledge discovery to extend the knowledge base.

3.3.1 Recommendation Assessment

As presented by (Fox et al., 2009), “the current

guideline development lifecycle does not provide

appropriate tools to assess their impact on clinical

practice”. The proposed system is able to evaluate the

decisions taken quantitatively (e.g. based on the

number of times the recommendation was followed)

and qualitatively (e.g. based on the patient outcomes

– when the results are successful or match the defined

decision criteria). This quantitative and qualitative

measurements are presented to clinicians during the

clinical decision making process to provide enriched

information of the given recommendations.

3.3.2 Patients’ Similarities

The system also applies similarity features between

different patients and their results to support

clinicians in their decision making process. For that,

the system uses different metrics to determine which

(clinical) parameters have higher impact when

HEALTHINF 2017 - 10th International Conference on Health Informatics

372

determining how similar a patient could be to a

retrospective patient (e.g. age range, TNM

classification etc.). In cases where the benefits and

harms of a specific treatment are not clear, clinicians

are able to consult previous similar patient cases and

their outcome before taking a decision. The previous

patient cases could be specific patient cases, or

‘model’ cases that summarise n past cases.

3.3.3 Knowledge Discovery

The experience acquired from the decisional history

may enable different type of knowledge acquisition.

Here we present the two types of knowledge

considered in our research.

Firstly, the information from a large number of

cases enables the adjustment of CPGs and protocols´

clinical conditions, e.g. in a form of a rule. For that,

the scope of the criteria is redefined based on the

cases where the given guideline-based

recommendation is being followed with successful

results. This is implemented using machine learning

techniques. For example, if a decision criterion is

parameter  ∈ 0.5, 1.5, after applying machine

learning techniques the system recognizes that the

recommendation is being successful only when ∈

0.8, 1.3. It also detects when a parameter, not

previously included into the clinical condition for the

decision making process, is determinant and should

be part of the existing decision rule.

Secondly, large number of non-compliant cases

with good or better results than the ones that follows

the CPGs may lead into an extension of the CPGs´

clinical conditions (e.g. rules) by generating ‘new’

branches. This ‘new’ branches may include

recommendations (treatment actions) that are not

considered in the available CPGs (e.g. “clinical trial”)

or may include recommendations that are in the

CPGs, but that are not considered for the given case.

This cases could make the knowledge base either

more restrictive when a rule becomes more precise,

but also could extend it with further procedures that

were not included in the knowledge base.

In both adjustment and extension cases, in order

to include the ‘new’ knowledge into the CDSS, the

system verifies if the outcomes are positives and

informs clinicians about its potential usage. If

approved, this knowledge is included into the

knowledge base for the CDSS (Figure 2).

Nevertheless, the system provides the information of

the recommendation source. This way clinicians are

aware if the recommendation is guideline based or

created automatically by the system based on the

recorded experience or patient similarity properties.

4 DESIGN IN DESIREE

This study is being performed in the context of a

European H2020 project, named DESIREE. In this

section we present DESIREE project (Section 4.1)

and the data flow diagram that represents our

methodology within DESIREE (Section 4.2).

Figure 2: Data flow diagram.

4.1 DESIREE

DESIREE aims to provide decision support on the

available therapy options by incorporating evidence

based guidelines and experience from previous cases

and outcomes. Hence, DESIREE goes beyond the

limitations of existing guideline-based decision

support systems. Such a system targets breast cancer

(BC) cases, which is one of the most common and

most deadly type of cancer affecting woman in the

EU countries, with more than 460,000 new cases and

130,000 deaths in 2012 (Ferlay et al., 2013).

The users of such system are medical domain

experts involved on breast units (BU) where patients’

diagnosis and treatment decisions are taken. Hence,

the system goal is to support BU during their weekly

meetings in their multidisciplinary decision making

process by providing not only CPGs based decision

support, but also additional information extracted

from previous cases over time.

4.2 DESIREE

The data flow diagram presented in Figure 2 is a high

level representation of DESIREE platform. Since

DESIREE is developed in the context of BC, in the

depicted figure, Breast Units (BU) are the clinical

experts that make the final decision. Here, we

describe each block presented in Figure 2, omitting

the blocks that correspond to the data presented in

Section 3.2.

Augmenting Guideline-based CDSS with Expertsâ

Z Knowledge

373

 Narrative Guidelines: In our methodological

approach, the starting point is the analysis of

representative and narrative CPGs used in BC

care.

 Computer Interpretable Guidelines:

Knowledge engineers extract the relevant

information from CPGs and formalize it in a

CIG. This covers the recommendations given

by guidelines for primary BC in several stages

of the whole treatment till the patient is

discharged. Hence, the CIG consider the

previous treatments and the outcomes of them

for the coming decision making action.

 Knowledge: The knowledge database stores

knowledge from the CIG or from the decisional

history exploitation´s “new” knowledge.

 Rule-based Engine: The rule-based engine is

able to generate recommendations having as

input the structured knowledge. Then, if patient

data fulfils the clinical condition, the rules are

fired and the engine generates one or more

recommendations.

 Knowledge Discovery: Based on a large set of

information stored in the decisional history, the

system is capable of retrieving knowledge as

discussed in Section 3.3.

5 DISCUSSION

The presented method overcomes some limitations of

current guideline-based CDSS by providing enriched

recommendations and additional information to

clinicians in order to support them best in their

decision making process. For that, we develop a new

information structure based on decisional events. A

decisional event stores the whole set of information

used in the decision making process, including the

consequences of the final decision, such as patient´s

outcomes (e.g. quality of life).

Here we present some of the potential benefits and

limitations of the proposed method. Firstly, the

system promotes the usage of CPGs. Additionally, it

assess the impact of the guidelines on clinical practice

(Section 3.3.1), which is one of the critical factors

detected by (Fox et al., 2009). Secondly, the system

flexibility enables the storage of additional valuable

information, such as the decision criteria, that could

be used to adjust or/and extend the clinical conditions

of the given protocols and CPGs over time (Section

3.3.3). This way it helps overcoming some of the

limitations of current CPGs presented in Section 2,

such as the impact of specific situations. This

diverges from the work done in other projects, such

as MobiGuide (Larburu et al., 2015), where the

guidelines are customized and made context-aware

beforehand during the knowledge engineering phase,

and not over time depending on previous cases.

Neither we focus on the discovery of temporal rules

from time-stamped data, like in (Sacchi et al., 2007).

Our study aims to discover rules from previous cases

tracking each case to assess the outcomes and

considering further information often not taken into

account in current CPGs, such as the implicit

knowledge of clinicians. Finally, the presented

method combines both the CPGs and the knowledge

generated automatically by the system based on their

experience, which overcomes the requirements

expressed by clinicians in (Miranda-Mena et al.,

2006): “clinicians want a system that combines the

protocol (or CPG) and their proper knowledge to

suggest treatments”.

6 CONCLUSIONS & FUTURE

WORK

The hypothesis of this research is that such approach

is more useful for clinicians, which expect a dynamic

system that not only considers available CPGs and

protocols, but also a system that is able to learn from

the stored information over time to provide enriched

decision support system.

In future work we aim to present among others the

following points: (i) the tools used to convert the

information acquired by experience into knowledge

to extend and adjust the CPGs and protocols; (ii) a

digital patient model ontology used for the CDSS, and

particularly for similarity purposes; (iii) the

methodology to assess the recommendations

applying different metrics (survival rate, overall well-

being, physical functioning etc.); and (iv) the

validation of the system in a representative number of

patients and the results.

ACKNOWLEDGEMENTS

This project has received funding from the European

Union’s Horizon 2020 research and innovation

programme under grant agreement No 690238.

REFERENCES

Adegboyega, T.O., Landercasper, J., Linebarger, J.H.,

Johnson, J.M., Andersen, J.J., Dietrich, L.L., Driscoll,

HEALTHINF 2017 - 10th International Conference on Health Informatics

374

C.D., Raghavendra, M., Madadi, A.R., Al-Hamadani,

M., Vang, C.A., Marcou, K.A., Hudak, J., Go, R.S.,

2015. Institutional review of compliance with NCCN

guidelines for breast cancer: lessons learned from real-

time multidimensional synoptic reporting. J. Natl.

Compr. Cancer Netw. JNCCN 13, 177–183.

Beeler, P.E., Bates, D.W., Hug, B.L., 2014. Clinical

decision support systems. Swiss Med. Wkly. 144,

w14073.

Bouaud, J., Séroussi, B., 2011. Revisiting the EBM

decision model to formalize non-compliance with

computerized CPGs: results in the management of

breast cancer with OncoDoc2. AMIA. Annu. Symp.

Proc. 2011, 125–134.

Cabana MD, Rand CS, Powe NR, et al, 1999. Why don’t

physicians follow clinical practice guidelines?: A

framework for improvement. JAMA 282, 1458–1465.

Chéreau, E., Coutant, C., Gligorov, J., Lesieur, B., Antoine,

M., Daraï, E., Uzan, S., Rouzier, R., 2011. Discordance

with local guidelines for adjuvant chemotherapy in

breast cancer: reasons and effect on survival. Clin.

Breast Cancer 11, 46–51.

Colantonio, S., Conforti, D., Martinelli, M., Moroni, D.,

Perticone, F., Salvetti, O., Sciacqua, A., 2008. An

intelligent and integrated platform for supporting the

management of chronic heart failure patients, in: 2008

Computers in Cardiology. Presented at the 2008

Computers in Cardiology, pp. 897–900.

Ferlay, J., Steliarova-Foucher, E., Lortet-Tieulent, J.,

Rosso, S., Coebergh, J.W.W., Comber, H., Forman, D.,

Bray, F., 2013. Cancer incidence and mortality patterns

in Europe: Estimates for 40 countries in 2012. Eur. J.

Cancer 49, 1374–1403.

Flottorp, S.A., Oxman, A.D., Krause, J., Musila, N.R.,

Wensing, M., Godycki-Cwirko, M., Baker, R., Eccles,

M.P., 2013. A checklist for identifying determinants of

practice: a systematic review and synthesis of

frameworks and taxonomies of factors that prevent or

enable improvements in healthcare professional

practice. Implement. Sci. IS 8, 35.

Fox, J., Patkar, V., Chronakis, I., Begent, R., 2009. From

practice guidelines to clinical decision support: closing

the loop. J. R. Soc. Med. 102, 464–473.

Grimshaw, J.M., Russell, I.T., 1994. Achieving health gain

through clinical guidelines II: Ensuring guidelines

change medical practice. Qual. Health Care 3, 45–52.

Hussain, S., Abidi, S.R., Abidi, S.S.R., 2007. Semantic

Web Framework for Knowledge-Centric Clinical

Decision Support Systems, in: Artificial Intelligence in

Medicine. Presented at the Conference on Artificial

Intelligence in Medicine in Europe, Springer, Berlin,

Heidelberg, pp. 451–455.

ICHOM – International Consortium for Health Outcomes

Measurement [WWW Document], n.d. URL

http://www.ichom.org/ (accessed 1.3.17).

Landercasper, J., Dietrich, L.L., Johnson, J.M., 2006. A

Breast Center review of compliance with National

Comprehensive Cancer Network Breast Cancer

guidelines. Am. J. Surg. 192, 525–527.

Larburu, N., Schooten, B. van, Shalom, E., Fung, N.,

Sinderen, M. van, Hermens, H., Jones, V., 2015. A

Quality-of-Data Aware Mobile Decision Support

System for Patients with Chronic Illnesses, in

Knowledge Representation for Health Care, Lecture

Notes in Computer Science. Springer International

Publishing, pp. 126–139.

Lebeau, M., Mathoulin-Pélissier, S., Bellera, C., Tunon-de-

Lara, C., Daban, A., Lipinski, F., Jaubert, D., Ingrand,

P., Migeot, V., REPERES Group, 2011. Breast cancer

care compared with clinical Guidelines: an

observational study in France. BMC Public Health 11,

45.

Leonard, R., Ballinger, R., Cameron, D., Ellis, P.,

Fallowfield, L., Gosney, M., Johnson, L., Kilburn, L.S.,

Makris, A., Mansi, J., Reed, M., Ring, A., Robinson,

A., Simmonds, P., Thomas, G., Bliss, J.M., 2011.

Adjuvant chemotherapy in older women (ACTION)

study - what did we learn from the pilot phase? Br. J.

Cancer 105, 1260–1266.

McGlynn, E.A., Asch, S.M., Adams, J., Keesey, J., Hicks,

J., DeCristofaro, A., Kerr, E.A., 2003. The Quality of

Health Care Delivered to Adults in the United States.

N. Engl. J. Med. 348, 2635–2645.

Mercuri, M., Gafni, A., 2011. Medical practice variations:

what the literature tells us (or does not) about what are

warranted and unwarranted variations. J. Eval. Clin.

Pract. 17, 671–677.

Miranda-Mena, T.G., Benítez U., S.L., Ochoa, J.L.,

Martínez-Béjar, R., Fernández-Breis, J.T., Salinas, J.,

2006. A knowledge-based approach to assign breast

cancer treatments in oncology units. Expert Syst. Appl.

31, 451–457.

Parks, R.M., Lakshmanan, R., Winterbottom, L., AL

Morgan, D., Cox, K., Cheung, K.-L., 2012.

Comprehensive geriatric assessment for older women

with early breast cancer – a systematic review of

literature. World J. Surg. Oncol. 10, 88.

Roshanov, P.S., Fernandes, N., Wilczynski, J.M., Hemens,

B.J., You, J.J., Handler, S.M., Nieuwlaat, R., Souza,

N.M., Beyene, J., Van Spall, H.G.C., Garg, A.X.,

Haynes, R.B., 2013. Features of effective computerised

clinical decision support systems: meta-regression of

162 randomised trials. BMJ 346, f657.

Sacchi, L., Larizza, C., Combi, C., Bellazzi, R., 2007. Data

mining with Temporal Abstractions: learning rules

from time series. Data Min. Knowl. Discov. 15, 217–

247.

Sackett, D.L., Rosenberg, W.M.C., Gray, J.A.M., Haynes,

R.B., Richardson, W.S., 1996. Evidence based

medicine: what it is and what it isn’t. BMJ 312, 71–72.

Schnitt, S.J., 1998. Microinvasive carcinoma of the breast:

a diagnosis in search of a definition. Adv. Anat. Pathol.

5, 367–372.

Séroussi, B., Laouénan, C., Gligorov, J., Uzan, S., Mentré,

F., Bouaud, J., 2013. Which breast cancer decisions

remain non-compliant with guidelines despite the use

of computerised decision support? Br. J. Cancer 109,

1147–1156.

Augmenting Guideline-based CDSS with Expertsâ

Z Knowledge

375

Séroussi, B., Laouénan, C., Gligorov, J., Uzan, S., Mentré,

F., Bouaud, J., 2013. Which breast cancer decisions

remain non-compliant with guidelines despite the use

of computerised decision support? Br. J. Cancer 109,

1147–1156.

Wöckel, A., Kurzeder, C., Geyer, V., Novasphenny, I.,

Wolters, R., Wischnewsky, M., Kreienberg, R., Varga,

D., 2010. Effects of guideline adherence in primary

breast cancer--a 5-year multi-center cohort study of

3976 patients. Breast Edinb. Scotl. 19, 120–127.

HEALTHINF 2017 - 10th International Conference on Health Informatics

376