SLR-TOOL

A Tool for Performing Systematic Literature Reviews

Ana M. Fernández-Sáez, Marcela Genero Bocco

ALARCOS Research Group, Department of Technologies and Information Systems, University of Castilla-La Mancha

Paseo de la Universidad, 4, 13071 Ciudad Real, Spain

Francisco P. Romero

SMILE Research Group, Department of Technologies and Information Systems, University of Castilla-La Mancha

Avda. Carlos III, s/n, 45071 Toledo, Spain

Keywords: Systematic Literature Review, SLR, Tool, Text mining.

Abstract: Systematic literature reviews (SLRs) have been gaining a significant amount of attention from Software

Engineering researchers since 2004. SLRs are considered to be a new research methodology in Software

Engineering, which allow evidence to be gathered with regard to the usefulness or effectiveness of the

technology proposed in Software Engineering for the development and maintenance of software products.

This is demonstrated by the growing number of publications related to SLRs that have appeared in recent

years. While some tools exist that can support some or all of the activities of the SLR processes defined in

(Kitchenham & Charters, 2007), these are not free. The objective of this paper is to present the SLR-Tool,

which is a free tool and is available on the following website: http://alarcosj.esi.uclm.es/SLRTool/, to be

used by researchers from any discipline, and not only Software Engineering. SLR-Tool not only supports

the process of performing SLRs proposed in (Kitchenham & Charters, 2007), but also provides additional

functionalities such as: refining searches within the documents by applying text mining techniques; defining

a classification schema in order to facilitate data synthesis; exporting the results obtained to the format of

tables and charts; and exporting the references from the primary studies to the formats used in bibliographic

packages such as EndNote, BibTeX or Ris. This tool has, to date, been used by members of the Alarcos

Research Group and PhD students, and their perception of it is that it is both highly necessary and useful.

Our purpose now is to circulate the use of SLR-Tool throughout the entire research community in order to

obtain feedback from other users.

1 INTRODUCTION

An SLR is a means of identifying, evaluating and

interpreting all available research relevant to a

particular research question, or topic area, or

phenomenon of interest (Kitchenham & Charters,

2007). These reviews differ from traditional

literature reviews in their rigorous and impartial

nature, which makes them of great scientific value.

The first methodology for conducting SLRs in

Software Engineering was presented by Barbara

Kitchenham (Kitchenham, 2004). This methodology

provides specific details of each of the phases and

activities evolved in carrying out an SLR. The

methodology was based on ideas taken from

medicine, a discipline in which SLRs are

indispensible, signifying that their realization is

considerably more mature in this area. Later, and

after three years of using the original methodology,

the same author proposed a new improved version of

the original method (Kitchenham & Charters, 2007).

SLRs have been gaining a significant amount of

attention from Software Engineering researchers

since 2004. They are becoming increasingly

important as the fundamental methodology through

which to contribute to the maturity of what is

denominated as “Evidence-Based Software

Engineering” (EBSE), whose main goal is ‘‘To

provide the means by which current best evidence

from research can be integrated with practical

experience and human values in the decision making

157

Fernández-Sáez A., Genero Bocco M. and Romero F. (2010).

SLR-TOOL - A Tool for Performing Systematic Literature Reviews.

In Proceedings of the 5th International Conference on Software and Data Technologies, pages 157-166

DOI: 10.5220/0003003601570166

 SciTePress

process regarding the development and maintenance

of software” (Dybå et al., 2005).

The relevance of SLRs is reflected in the

numerous papers that have been written since 2004

which either present SLRs of a specific subject or

deal with methodological aspects. For example,

(Brereton, et al., 2007; Turner, et al., 2008) present

lessons learned which were obtained after

performing several SLRs by following the proposed

methodology. (Dieste & Padua, 2007) present a

means to develop optimal search strategies that

retrieve as much relevant information as possible,

while maintaining low costs and effort. (Babar &

Zhang, 2009) present preliminary results from

interviews with researchers which were carried out

with the goal of independently exploring the

experiences and perceptions of the practitioners of

SLRs in order to gain an in-depth understanding of

various aspects of SLRs as a new research

methodology in Software Engineering. In

(Kitchenham, et al., 2009) an observer-participant

case study is used to analyse the impact of limited

search procedures for SLRs. In (Kitchenham, et al.,

2009) an SLR of SLRs is presented whose objective

is to review the current status of EBSE since 2004.

20 SLRs published between 2004 and 2007 are

analysed in this paper.

Further evidence of the relevance that SLRs are

taking on as a research methodology in Software

Engineering is that from 2005 onwards the

Information and Software Technology Journal has

included SLRs as a new type of paper for

submission. Moreover, since 2007 there have been

special sessions related to SLR issues at the EASE

and ESEM conferences.

While there is an established methodology for

conducting SLRs, most of the authors of papers

containing SLRs stress the difficulty of carrying

them out, which is particularly caused by the low

amount of flexibility of searches that most digital

libraries provide and the lack of a tool to support the

entire process of SLRs which would reduce the time

and resources required for effectively and efficiently

carrying out SLRs without compromising their

quality (Babar & Zhang, 2009).

The objective of this paper is to present the SLR-

Tool that we have designed and implemented to

support each of the phases in the SLR process. The

main advantage of this tool is that, unlike other

existing tools, it is free, and reduces the effort

required to carry out the SLR manually.

With regard to the SLR-Tool’s functionality, it

has the following advantages:

 It can store data related to each of the activities

in each of the phases of the process used to

perform SLRs.

 It allows the searches to be refined by using text

mining techniques.

 It permits the definition of classification scheme

which helps the researcher to perform data

synthesis and analysis.

 It uses text mining techniques to cluster the

review documents by using the similarities

among them.

 It exports all the data collected in the review

process to Excel files in table or graphic

formats. It also permits the export of all the

references of the documents uploaded in the tool

to the format accepted by bibliographic

packages such as EndNote, BibTeX and Ris.

The remainder of the paper is structured as follows:

Section 2 presents a summary of the tools that

currently exist to perform SLRs, emphasizing the

differences between them and the tool that we

propose in this paper. In Section 3 shows the

processes for performing SLRs. Section 4 presents

design and implementation details of SLR-Tool and

Section 5 shows an example of how the tool has

been used. Finally, Section 6 presents some

conclusions and future work.

2 RELATED WORK

To the best of our knowledge, several tools covering

some or all of the phases in the SLR process

presented in (Kitchenham & Charters, 2007) exist,

and these are summarized below:

 EPPI-Reviewer (EPPI-Reviewer, 2010). This

web tool can be used by various researchers to

carry out a collaborative systematic review.

Besides supporting bibliographic management,

and inclusion and exclusion criteria

management, EPPI-Reviewer focuses most of

its functionality on plotting results, generating

reports and applying certain meta-analysis

techniques.

 TrialStat's SRS software (TrialStat, 2010) is a

commercial tool. It is necessary to pay a

substantial amount to obtain a license to use this

tool.

 Tools to manage bibliographies also exist. Some

of these act as metasearchers, which allow

searches to be made in digital libraries such as

ACM or IEEE, or in reference managers such as

CiteSeer. They also permit the searches to be

ICSOFT 2010 - 5th International Conference on Software and Data Technologies

158

refined with their own searchers. The problem

with this type of tools is that they do not support

the complete process involved in SLR process,

since they only support the search for candidate

papers for primary studies. Some examples of

these tools are presented in (JabRef, 2010;

Skidmore, 2002).

 There are also numerous tools that allow

empirical data to be integrated through meta-

analysis, thus enabling general conclusions

about the empirical data found in SLRs to be

obtained.

Those tools that support the whole SLR process

need a license for their use, while those tools whose

functionality is limited to part of the SLR are free.

There is thus a need to develop an SLR-Tool in

order to provide a tool for performing SLRs that is

freely available and which can overcome some of

the limitations of digital library search engines, such

as the difficulty involved in defining complex search

strings, or in allowing searches to be made in

different fields of documents such as the title, the

abstract, the keywords or the full text.

SLR-Tool is available on the

http://alarcosj.esi.uclm.es/SLRTool/ website, and

can therefore be used by researchers from the

Software Engineering community, from whom we

hope to obtain feedback in order to adapt the tool to

the real needs of researchers.

3 SLR PROCCESS

According to (Kitchenham & Charters, 2007) an

SLR is an evaluation and interpretation of all

existing research that is relevant to a specific

research question, an area of knowledge or a

phenomenon of interest. SLRs aim to provide a fair

assessment of a research topic through a reliable,

rigorous and auditable methodology.

The importance of SLRs lies in their impartiality.

Many researchers carry out small research studies

that may initially appear to be SLRs. However,

unless it is a thorough and impartial process the

results may lack scientific value.

The three main phases of an SLR consist of

planning, conducting and reporting the review, each

of which will be expalined as follows.

3.1 Planning the Review

The first action that must be performed in this phase

is that of identifying the need to carry out a review,

which arises from researchers’ needs to summarize

all existing information on a phenomenon in a

thorough and impartial way.

Sometimes an organisation requires information

about a specific topic but does not have the time or

expertise to perform an SLR itself. In such cases it

will commission researchers to perform an SLR of

the topic. When this occurs the organisation must

produce a commissioning document specifying the

work required.

It is then neccesary to specify the research

questions, which is the most important part of any

SLR. The review questions drive the entire

systematic review methodology:

 The search process must identify primary

studies that address the research questions.

 The data extraction process must extract the

data items needed to answer the questions.

 The data analysis process must synthesise the

data in such a way that the questions can be

answered.

The following step should be that of developing a

review protocol. Its implementation is necessary to

avoid the possibility of researchers’ bias. The

components of a protocol include all the elements of

the review plus some additional planning

information:

 Background, and reason for carrying out the

review.

 The research questions that the review intends

to answer.

 The strategy used to search for primary studies,

including the terms and resources (digital

libraries, specific journals and conference

proceedings) that will be searched.

 Study selection criteria. These criteria are used

to determine which studies will be included in

or excluded from the review.

 Study selection procedures. These indicate how

researchers should apply the selection criteria,

i.e. how many people have to evaluate each

potential primary study, how to resolve

disagreements, and so on.

 Study quality assessment checklists and

procedures. The researchers should develop

quality checklists to assess the individual

studies.

 Data extraction strategy. This defines how the

information required from each primary study

will be obtained. If the data require

manipulation or assumptions and inferences to

be made, the protocol should specify an

appropriate validation process.

 Synthesis of extracted data. Define the synthesis

strategy.

SLR-TOOL - A Tool for Performing Systematic Literature Reviews

159

 Dissemination strategy. Where and when the

results are intended to be presented. This point

is included only if it is not specified in the

review commission document.

 Project timetable. This should define the review

schedule.

Finally, the review protocol must be evalueted. If

sufficient funding is available then it is advisable to

ask an independent panel to review the protocol.

3.2 Conducting the Review

In this phase, it is first necessary to identify the

search with regard to the research in question, thus

generating a search strategy and documenting the

search. Strategies are often iterative, and benefit

from:

 Previous research aimed at identifying existing

SLRs and assessing the volume of potential

studies.

 An attempt to use various combinations of

research terms to discover the question. One

approach is to divide the question into parts,

find synonyms, abbreviations, or to use logical

operators like "AND" and "OR", etc. The most

common places in which to carry out this search

are reference lists, magazines, newspapers,

conferences, research and Internet records.

 Consult experts in the field.

The study selection is then carried out, i.e. the

primary studies are selected. This step seeks to

assess the relevance of the primary studies with

regard to the research questions. Final decisions for

inclusion or exclusion should be made after

reviewing the full text, and it is important to indicate

the reason for exclusion. It is then necessary to

analyze the reliability of the listing decisions. Two

or more researchers must evaluate each document,

and discuss and resolve their differences. This can

be done by evaluating all the documents, or by

selecting a random sample to serve as an example.

The following step is to assess the quality of the

primary studies in order to determine the individual

importance of each study, interpret the strength of

inferences, recommend future research, etc.

Researchers can use hierarchies of evidence to

restrict the primary studies that they wish to include

in their systematic review.

It is then necessary to design a means to extract

the data that connect the information from the

primary studies. At this point, it is advisable to

create a classification scheme and classify the

primary studies in accordance with this.

Finally, we sintethize the data, summarising the

results obtained from of the primary studies.

3.3 Reporting the Review

The final phase involves writing up the results of the

review and circulating them to potentially interested

parties. It is first important to prepare a report, which

should contain all information related to the various

phases of the review and a summary of all the results

obtained. This report may then be disseminated in

official publications such as conferences, magazines,

or workshops, or in other forms such as web pages,

etc.

4 SLR-TOOL DESCRIPTION

This section presents some details related to the

design and implementation of SLR-Tool. SLR-Tool

has been developed by using the JAVA

programming language. The integrated development

environment (IDE) chosen has been Eclipse. The

tool runs under the Windows operative system (XP

version and latter). SLR-Tool is a multi-language

tool in which both English and Spanish interfaces

are available.

The development of SLR-Tool has followed the

Rational Unified Process (RUP) (Jacobson, et al.,

1999). The modeling language used is UML 2.0

(OMG, 2003).

The tool has been designed through a three-layer

architecture (Larman, 2001), which isolates each of

the main elements of operations so that the

presentation is independent of processing rules and

business logic which are, in turn, independent of

data. This model significantly reduces maintenance

costs and increases long term functional flexibility.

The functionality of the tool is summarized in the

activity diagram shown in Figure 1. The stages may

appear to be sequential, but it is important to

recognise that many of the stages involve iteration.

In particular, many activities are initiated during the

planning of the review, and refined when the review

itself takes place.

The filled in activities are those activities that

will be carried out externally to the tool, and the

double circled activities are those activities which

are characteristic of the tool itself and were not

proposed by (Kitchenham & Charters, 2007), and

are therefore our own contribution.

ICSOFT 2010 - 5th International Conference on Software and Data Technologies

160

Figure 1: Activity diagram of SLR-Tool.

SLR-Tool enables all the information needed to

carry out the SLR process proposed in (Kitchenham

& Charters, 2007) to be stored and managed. This

differentiates it from other existing tools that simply

manage the bibliographic stages as with the

previously mentioned JabRef (JabRef, 2010) or

similar.

SLR-Tool allows the searches to be refined by

using text mining techniques. The number of

relevant papers can therefore be reduced by filtering

only those that fit with the full search string. Most

search engines in the major digital libraries have

limitations in defining complex search strings or do

not allow all the fields of the document to be

searched. The refinement of the searches was carried

out by using the Lucene (Lucene, 2009) search

engine. It is worth noting that SLR-Tool is not a

metasearch but that the search for documents is

performed manually and independently of the tool,

and once the documents have been obtained, they

are uploaded into SLR-Tool.

The tool provides three possible ways in which

to import or store the documents found:

 Manually. The user introduces the metadata

related to the document being imported, along

with the document in PDF format. If the

abstract field is not added, the tool will read the

PDF file and try to locate it.

 From EndNote, BibText, Ris files. The user

introduces an EndNote/BibTex/Ris reference

archive (from which the metadata are extracted)

and the document in PDF. As with the first

option, if the abstract has not been recovered

from the reference file, it can be extracted from

the PDF file.

 Automatically from a PDF file. In this case the

user only introduces the document in PDF

format and the tool attempts to extract the

metadata by analyzing the file.

Furthermore, when the documents are loaded

into the tool, it automatically detects the existence of

duplicates, removing the copy that the user indicates,

and keeping count of duplicated documents.

It is worth highlighting that SLR-Tool permits a

classification scheme to be defined that helps the

researcher to perform data synthesis and analysis.A

classification scheme can be used to define the

categories and subcategories required to classify the

primary studies in order to sinthetize the results of

the review. The classification scheme that can be

created with this tool permits a tree of two levels to

be created, i.e., it either permits categories that the

possible values depend on when a document is

assigned, or categories with sub-categories, in which

case the values to be assigned depend on the latter.

Once all the primary studies have been classified

with the classification scheme defined, SLR-Tool

can generate tables and charts to summarize the data.

The researcher will therefore have visual access to

this data thus allowing him/her to form a rapid and

easy idea of the situation, therefore facilitating the

synthesis of data and the extraction of conclusions.

If it is necessary to carry out meta-analysis to

integrate empirical data, this must be done by using

SLR-TOOL - A Tool for Performing Systematic Literature Reviews

161

Figure 2: Search strategy.

another tool, since SLR-Tool does not have this

functionality.

SLR-Tool exports all the data collected in the

review process to Excel file sheets and the charts to

PDF files. This makes the data more manageable,

allowing them to be used in any documents or

papers in which it is intended to report the

conclusions obtained. SLR-Tool also allows all the

bibliographic data from the primary studies

uploaded in the tool to be exported to the format

accepted by bibliographic packages such as

EndNote, BibTeX and Ris. This facilitates the use of

these references in subsequent publications.

Finally, it is important to note one of the tool’s

other functionalities. It uses text mining techniques

to cluster the documents by using the similarities

among them, highlighting key words that identify

each group of documents. This is of assistance when

it is necessary to know whether the documents found

in the search are really related to the subject under

study, thus permitting the exclusion of those

documents that centre on a subject, which is too far

removed from the subject under study.

5 EXAMPLE

An example of the use SLR-Tool to perform an SLR

on "Quality of UML models "(Genero et al., 2009) is

shown below.

The data related to the planning phase recommended

in (Kitchenham & Charters, 2007) was first stored in

the tool. As is shown in Figure 3, general data such as

the title of the review, the dates of commencement

and completion, the background, and so on were

stored.

Figure 3: General data.

We then posed the research questions that were

intended to be answered through the results obtained

by the SLR (Figure 4). Once the results had been

analyzed in later stages, answers to these questions

were obtained and stored in the attachment column.

Figure 4: Research questions.

At the beginning of the phase which involved

conducting the review, it was necessary to define the

search strategy that had to be followed. This was

done by first loading the different digital libraries to

be used as bibliographic sources (top of Figure 2).

SLR-Tool automatically generates a source of

information called gray literature in which it is

ICSOFT 2010 - 5th International Conference on Software and Data Technologies

162

Figure 5: Execution phase.

possible to add all those relevant documents that are

known by experts but have not been returned by any

search.The next step was to add the search strings

that were originally intended for the search engines

of each digital library. One or more search strings

are possible. The adjustment of each string then had

to be added to each of the search engines defined

above, using each of their operators. It is also

important to add the date on which the search is

conducted (bottom of Figure 2).

The inclusion and exclusion criteria are defined

here (

Figure 6). The quality criteria are also defined

in a similar manner.

Figure 6: Inclusion/Exclusion criteria.

The search was conducted independently of SLR-

Tool, and the documents obtained were stored in the

tool using one of three possible methods previously

described by the tool. Once the documents had been

loaded into the tool (Figure 5), the search was

refined.

For example, the IEEE Computer search engine

presents the two fields that can be filled in to carry

out the search with logical operators as being an

“advanced search” (

Figure 7). However, this search

cannot be limited to the document’s abstract. In

highly studied research themes many more

documents than those desired are therefore obtained,

since the terms being sought may appear in the

complete text as introductory or isolated terms,

which do not ensure the document’s relationship

with the subject of interest. It is therefore of interest

to limit the search for relevant terms to the abstract.

Figure 7: Advanced search in IEEE Computer.

Our original search string was: (UML or unified

modeling language) and (representation or diagram

or model) and (quality or consistency or

maintainability or understandability or completeness

or comprehension or comprehensability or testability

or defect or effectivennes or complexity or

readability or efficiency or validation or verification

or layout).

In this search the search engine returned 109

documents. These were all loaded into the tool, and

the search was refined by using the tool to search for

SLR-TOOL - A Tool for Performing Systematic Literature Reviews

163

the search string in the abstract (Figure 8). The

number of documents was therefore reduced to 66.

Figure 8: Refine the search.

Table 1: Example of the classification scheme.

Category Subcategory Value

Type of

quality

Syntactic Correctness

Semantic

Consistency

Completeness

Correctness

Pragmatic

Maintainability

Analyzability

Understandability

Testability

Functionality

Executability

Reusability

Complexity

Dependability

Type of

diagram

Class diagrams

Sequence diagrams

Activity diagrams

Use case diagrams

Statechart diagrams

Collaboration diagrams

Component diagrams

Package diagrams

Interaction overview

diagrams

UML models

UML 2.0 new diagrams

View

Checklist, rules, modeling

conventions, and guidelines

We then defined the classification scheme used to

classify the documents that were considered to be

relevant, as was explained in Section 5.

Table 1

shows part of the classification scheme used.

We next read each of the 399 documents found

(from IEEE and from the other search engines), to

decide which we considered to be relevant, bearing

in mind the inclusion/exclusion criteria defined

during the planning phase. The tool allows us to

indicate which review of the document led to its

exclusion (Figure 9): the first (in which only the title

and abstract of the document were used) or the

second (using the full text).

Figure 9: Exclusion of a document.

All those documents that were not excluded from the

review (193 in total) were considered as primary

studies. These were then classified in accordance

with the classification scheme defined (

Figure 10).

We also decided how each of the documents fulfils

the quality criteria defined in the planning stage.

Figure 10: Classification of a document.

Finally, we used the Report Results option to

automatically generate tables and charts such as:

 Summary of documents found by each search

engine, detailing those that were initially found,

were eliminated owing to repetition and were

excluded.

 Summary of classified documents for each

category or subcategory of the defined

classification scheme. Figure 11 shows an

example of this type of charts generated by the

tool.

 Summary of compliance of the documents with

each defined quality criterion.

 Summary of documents found by year and by

document type (journal, book, etc).

6 CONCLUSIONS

The main contribution of this paper is the

presentation of SLR-Tool, a freely-available tool for

performing SLRs by following the process proposed

in (Kitchenham & Charters, 2007).

SLR-Tool has been tested through its use in

(Genero, et al., 2009). SLR-Tool has also been used

by the university teachers who are members of the

ICSOFT 2010 - 5th International Conference on Software and Data Technologies

164

ALARCOS Research Group, and it is currently

being used by our Ph.D students.

Figure 11: Charts generated by SLR-Tool.

The main advantage of the SLR-Tool is that its

use implies a significant reduction in effort when

compared to that involved in carrying out all the

activities manually. Although it is rather difficult to

upload all the primary studies included in the

review, the tool allows the search among the

documents found to be refined, thus reducing the

number of papers with which it is necessary to work.

The tool also permits the definition of a

classification scheme, which facilitates the

classification of documents. The tool uses this

classification to automatically generate charts and

graphs that summarize the results, so that if there is a

change in the classification of a document it will not

be necessary to do any additional work to update the

tables and graphs. Finally, also note that it is

possible to export the primary studies to files in

different formats such as EndNote, BibTeX or Ris,

thus permitting their use as bibliographical

references in other documents.

Possible improvements that could be made to the

tool are:

 The addition of a wizard to adapt the search

strings to each bibliographic source.

 Allowing the initial search for papers to be

made directly in the tool, by directly accessing

the search engines of the bibliographic sources.

 Allowing SLRs to be carried out in a

collaborative manner.

 Allowing the tool to be used via the Web.

 Allowing document metadata to be imported

from Excel files.

 Allowing review data to be exported to open

source formats.

 The addition of a “scheduling module” to plan

the timetable of each review phase in advance.

 Making the tool available in other languages,

and not only English and Spanish.

All appropriate suggestions made by researchers

who have downloaded and used the tool will also be

incorporated.

ACKNOWLEDGEMENTS

This research is part of the following projects:

PEGASO/MAGO (MICINN and FEDER, TIN2009-

13718-C02-01), EECCOO (MICINN

TRA2009_0074), IDONEO (JCCM PAC08-0160-

6141), MECCA (JCMM PII2I09-0075-8394).

REFERENCES

Babar, M. A., & Zhang, H. (2009). Systematic Literature

Reviews in Software Engineering. In the Empirical

Software Engineering and Mesurement (ESEM 2009),

Florida, USA.

Booch, G., Rumbaugh, J., & Jacobson, I. (1998). Unified

Modeling Language User Guide: Adisson Wesley.

Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M.,

& Khalil, M. (2007). Lessons from applying the

systematic literature review process within the

software engineering domain. Journal of Systems and

Software, 80(4), 571-583.

Dieste, O., & Padua, A. G. (2007). Developing Search

Strategies for Detecting Relevant Experiments for

Systematic Reviews. In Proceedings of the First

International Symposium on Empirical Software

Engineering and Measurement (pp. 215-224): IEEE

Computer Society.

Dybå, T., Kitchenham, B. & Jørgensen, M. (2005).

Evidence-based Software Engineering for

Practitioners. IEEE Software, 22(1), 58-65.

EPPI-Reviewer. (2010). EPPI-Center. Retrieved the 10th

of February of 2010, from

http://epi.ioe.ac.uk/cms/Default.aspx?tabid=184

Genero, M., Fernandez, A. M., Nelson, H. J., Poels, G., &

Piattini, M. (2009). A Systematic Literature Review

on the Quality of UML Models. Submitted to Journal

of Database Management (JDM).

SLR-TOOL - A Tool for Performing Systematic Literature Reviews

165

JabRef. (2010). JabRef: Reference Manager. Retrieved the

11th of February of 2010, from

http://jabref.sourceforge.net

Jacobson, I., Booch, G., & Rumbaugh, J. (1999). The

Unified Software Development Process.

Kitchenham, B. (2004). TR/SE-0401. Procedures for

performing systematic reviews. Keele University.

Kitchenham, B., Brereton, O. P., Budgen, D., Turner, M.,

Bailey, J., & Linkman, S. (2009). Systematic literature

reviews in software engineering - A systematic

literature review. Information and Software

Technology, 51(1), 7-15.

Kitchenham, B., Brereton, P., Turner, M., Niazi, M.,

Linkman, S., Pretorius, R., & Budgen, D. (2009). The

Impact of Limited Search Procedures for Systematic

Literature Reviews - An Observer-Participant Case

Study. In the Empirical Software Engineering and

Mesurement (ESEM 2009), pp. 336-345.

Kitchenham, B., & Charters, S. (2007). EBSE-2007-01.

Guidelines for performing Systematic Literature

Reviews in Software Engineering. Keele University.

University.

Larman, C. (2001). Applying UML and Patterns: An

Introduction to Object-Oriented Analysis and Design

and the Unified Process: Prentice-Hall.

Lucene. (2009). The Apache Software Foundation.

Retrieved the 12th of March of 2009, from

http://lucene.apache.org/

OMG. (2003). MDA Guide (Vol. Version 1.0.1.): from

http://www.omg.org/docs/omg/03-06-01.pdf.

Skidmore, B. (2002). CCOHTA expands the use use of

bibliographic software in systematic reviews: A new

role of Reference Manager. In the Annual Meeting

International Society of Technology Assessment in

Health Care, Green Valley Crescent, Ottawa, 18,

abstract nº 289.

Romero, F. P., Olivas, J. A., & Garcés, P. J. (2006). A Soft

Approach to Hybrid Models for Document Clustering.

In the Information Processing and Management of

Uncertainty in Knowledge-based Systems (IPMU

2006), Paris Les Cordeliers, France.

TrialStat. (2010). TrialStat Corporation. Retrieved the 9th

of February of 2010, from http://www.trialstat.com

Turner, M., Kitchenham, B., Budgen, D., & Brereton, O.

P. (2008). Lessons learnt undertaking a large-scale

systematic literature review. In the Evaluation and

Assessment in Software Engineering (EASE 2008).

ICSOFT 2010 - 5th International Conference on Software and Data Technologies

166