An Application of OSSpal for the Assessment of Open Source Project

Management Tools

Hugo Carvalho de Paula

and Jorge Bernardino

1,2 a

Polytechnic of Coimbra, ISEC, Rua Pedro Nunes, Quinta da Nora, 3030-199 Coimbra, Portugal

CISUC - Centre of Informatics and Systems of University of Coimbra, Pinhal de Marrocos, 3030-290 Coimbra, Portugal

Keywords: Software Assessment, Project Management.

Abstract: Projects are a necessity within any competitive business, and as the execution of complex projects becomes

the norm, so grows the need for advances in project management. The use of project management tools is key

towards taming said complexity. There are many such tools available; the current challenge resides in picking

the right one. In this paper, we evaluate three different tools - OpenProject, dotProject, and Odoo - using the

OSSpal methodology.

1 INTRODUCTION

Project management (PM) is the application of

knowledge, skills, tools, and techniques to project

activities to meet the project requirements (Project

Management Institute, 2017, p. 10). One of the

activities in a project is PM itself; therefore, a tool

must be applied to PM.

A natural question is how to properly determine

which tool to use. First, we recognize that PM is a

complex task, and that various kinds of software are

well-suited for specific subtasks in PM. We would

like to evaluate, then, general software; that is, tools

that assist PM throughout the whole project lifecycle.

Second, we prefer open-source software. The debate

between open-source and proprietary is beyond this

paper’s scope, but interested readers might want to

read papers like (Wheeler, 2015) or (Boulanger,

2005) for more general research, and (Abramova,

Pires, & Bernardino, 2016) for research that’s also

related to PM tools.

To answer the question, we turn to the field of

Open Source Software Assessment. It is a field that

has sprung up in response to the several challenges

brought by the increasing adoption of Open Source in

various enterprise projects. Practitioners in this field

have developed Open Source Software Assessment

Methodologies (OSSAMs, or just OSSAM in the

https://orcid.org/0000-0001-9660-2011

singular), well-organized collections of methods,

suitable for the complete task of assessment.

We apply a well-known OSSAM, the OSSpal

methodology (Wasserman, et al., 2017). The OSSpal

methodology is the successor project to the Business

Readiness Rating (BRR) (OpenBRR, 2005). OSSpal

methodology combines quantitative and qualitative

measures for evaluating software in several

categories, resulting in a quantitative value that

allows the comparison between tools (Wasserman, et

al., 2017).

In the OSSpal website (OSSpal software list,

2019), one can find several evaluations published. In

addition to those, we cite (Pereira, Sousa, Santos, &

Bernardino, 2018) and (Ferreira, Pedrosa, &

Bernardino, 2018) as related work, as they use

OSSpal to evaluate software in areas such as Data

Mining and e-Commerce.

We apply OSSpal to assess the area of project

management tools, focusing on three different tools

for PM: OpenProject (OpenProject main page, 2019),

dotProject (dotProject main page, 2019) and Odoo

(Odoo main page, 2019). This paper presents the

results of this evaluation.

The paper is organized as follows. Section 2

presents the three PM tools that will be evaluated.

Section 3 describes the OSSpal methodology. Section

4 presents the results of the evaluation. Finally,

section 5 presents the conclusions.

Carvalho de Paula, H. and Bernardino, J.

An Application of OSSpal for the Assessment of Open Source Project Management Tools.

DOI: 10.5220/0008412404110417

In Proceedings of the 15th International Conference on Web Information Systems and Technologies (WEBIST 2019), pages 411-417

ISBN: 978-989-758-386-5

411

2 OPEN SOURCE PROJECT

MANAGEMENT TOOLS

Project management tools assist the manager in

estimating, controlling, and tracking projects. Such

tools can have a profound impact on the quality of

project management and, consequently, on the quality

of the project itself. These tools can provide several

features that assist the manager with vital project

decisions, such as reporting, scheduling control, and

automating some day-to-day activities.

Even within the open-source and general tools,

there are still many choices for the practitioner. Thus,

we only present here the specific software that was

studied in detail: OpenProject, dotProject and Odoo.

2.1 OpenProject

OpenProject is a web-based project management

system for location-independent team collaboration.

It is an Open Source software registered under the

GNU General Public Version 3 License and is

continuously developed by an open source

community.

Some of its features are:

 Milestones management;

 Issue and bug tracking;

 Wiki;

 Document management;

 Forum.

 Time tracking;

 Project newsfeed.

OpenProject supports both traditional and agile

processes. Its agile support includes the usage of

taskboards to break releases into individual tasks and

perform status updates with “drag’n’drop” actions. In

Figure 1, we can see a screenshot of OpenProject in

action.

Figure 1: OpenProject interface.

2.2 dotProject

dotProject is an Open Source project management

tool distributed under the GNU General Public

Version 2 License. It is a web tool, written in PHP,

that uses a MySQL database. The goal of dotProject

as a tool is to unify the important information of the

project, presenting an overview of the tasks and

responsibilities.

There is no business behind the project, it is

managed, maintained, and developed by a group of

volunteers and by the users themselves who

furthermore provide software support.

The current version of the software is available on

GitHub, with the documentation and guidelines on

the proper use of the resources still in the process of

being moved to a newer page. Figure 2 presents the

interface of the tool in action.

Figure 2: dotProject interface.

2.3 Odoo

Odoo is an integrated ERP system, whose main

characteristics are: it is an open source system; it is

cross-platform, since through any web browser you

can access its interface; it is easy to handle; it interacts

with other applications, with PDF viewing, import

and export of Microsoft Office documents, and also

compatibility, for example, with Google Maps.

Odoo is not only a simple management platform,

it encompasses sales and project management in one

tool, and enriches it with a variety of functions, such

as MRP, Point of Sale, and e-commerce applications,

in order to provide a universal solution that can help

companies manage all kinds of operations.

Figure 3 showcases Odoo, as it is used for project

management.

Figure 3: Odoo interface.

WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies

412

3 OSSpal METHODOLOGY

Based on the knowledge acquired through the study

of the tools previously explained, we now turn to the

OSSpal methodology explanation.

OSSpal has emerged as a successor to Business

Readiness Rating (OpenBRR), to provide a reliable

and unbiased source for evaluating open source

software (Marinheiro et al., 2015). The goal is to be

an open, comprehensive and standard evaluation

model that is reliable, widely used and adjustable.

OSSpal combines quantitative and qualitative

assessment measures that allow to identify which

software has the best score. It can help companies,

government agencies and other organizations find

high quality open source software (Wasserman, et al.,

2017).

The OSSpal methodology works by scoring

software across seven different categories, and then

computing a total score for each software, based on

the individual category scores and previously

determined weights.

The categories of OSSpal methodology are the

following:

 Functionality: How well will the software meet

the average user’s requirements?

 Operational Software Characteristics: How

secure is the software? How well does the

software perform? How well does the software

scale to a large environment? How good is the

UI? How easy to use is the software for end-

users? How easy is the software to install,

configure, deploy, and maintain?

 Support and Service: How well is the software

component supported? Is there commercial

and/or community support? Are there people

and organizations that can provide training and

consulting services?

 Documentation: Is there adequate tutorial and

reference documentation for the software?

 Software Technology Attributes: How well is

the software architected? How modular,

portable, flexible, extensible, open, and easy to

integrate is it? Are the design, the code, and the

tests of high quality? How complete and error-

free are they?

 Community and Adoption: How well is the

component adopted by community, market,

and industry? How active and lively is the

community for the software?

 Development Process: What is the level of the

professionalism of the development process

and of the project organization as a whole?

The OSSpal uses a very similar calculation as the

BRR. We can exemplify the OSSpal methodology

calculation with an image from that method (see

Figure 4):

Figure 4: BRR score calculation (OpenBRR, 2005).

This methodology is composed of four phases:

1. First Phase: it is necessary to identify

a software component list to be analyzed,

to measure each component in relation to

the evaluation criteria and removing

from the analysis any software

component that does not satisfy the user

requirements;

2. Second Phase: it should be attributed

weights for the categories and for the

measures:

a. Assign a percentage of

importance to each category,

totaling 100%;

b. For each measure within a

category, it is necessary to rank

the measure in accordance with

its importance;

c. Foreach measure within a

category assign the importance

by percentage, totaling all the

measures 100% of the category.

3. Third Phase: gather data for each

measure used in each category and

calculate its weighting in a range

between 1 to 5 (1 - Unacceptable; 2 -

Poor; 3 – Acceptable; 4 -Very Good;

and, 5 - Excellent);

4. Fourth Phase: the qualification of the

category and the weighting factors

should be used to calculate the OSSpal

final score.

The category ‘Functionality’ is calculated

differently from the others. In this category is

An Application of OSSpal for the Assessment of Open Source Project Management Tools

413

intended to analyze and evaluate the characteristics

which the tools have or should have. The method to

assess this category is as follows:

A. Set down the characteristics to

analyze, scoring them from 1 to 3 (less

important to very important);

B. Classify the characteristics in a

cumulative sum (from 1 to 3);

C. Standardize the prior result to a scale

from 1 to 5.

Therefore, the Functionality category will have

the following scale:

 Under 65%, Score = 1 (Unacceptable)

 65% -80%, Score = 2 (Poor)

 80% -90%, Score = 3 (Acceptable)

 90% -96%, Score = 4 (Good)

 Over 96%, Score = 5 (Excellent).

4 EVALUATION

To follow up and start the evaluation following the

OSSpal methodology, we carried out research,

installed the tools, and proceeded to the basic use,

from a user's perspective. From this process, we

obtain several results, to be detailed in the remainder

of this section.

For the first phase, after the initial research, we

separated the tools we found into three groups, based

on the following requirements:

 Good integration with external software. First,

we determine the validity of this requirement

from (Akita, 2015). Then we decided to use

Microsoft Excel as a baseline, given its

ubiquity in the corporate world. Based on this,

the software with the best integration was

OpenProject, due to its custom synchronization

feature (Lindenthal, 2019);

 Good integration with internal software. This

requirement appears as a counterpart to the

first, in such a way that we represent all the

tools that stand out for the good integration.

Odoo stood out among the others, as it is

actually an entire ERP, which happens to

include a project management tool.

 PMBOK (PMI, 2017) compliance. The

PMBOK is one of the leading guides in the

world of project management, so it is only

natural to require software that incorporates its

lessons directly. We found dotProject as the

highlight here, due to the research found in

(Gonçalves R. Q., 2018).

The result of the first phase is, therefore, the

three tools that we will evaluate in more detail;

OpenProject, dotProject, and Odoo.

As part of the result corresponding to the second

phase of the methodology, we list in Table 1 the

weights of the criteria to be evaluated.

Table 1: Weights of each OSSpal evaluation category.

most popular, followed by OpenProject,

and finally dotProject.

 Support and Service: Both Odoo and

OpenProject have corporations that

provide support, but they are small.

dotProject still has some support, but it's

not much.

 Development Process: dotProject is

currently developed by few people,

without much organization. The other

tools are products of a company, so it has

a high value in this criterion.

Following the methodology, we then proceed to

phase 4, where we calculate the final score of each

tool, applying the weights previously defined.

OpenProject = 3 x 0.3 + 4 x 0.3 + 3 x 0.1 + 4 x 0.1

+ 3 x 0.05 + 3 x 0.1 + 4 x 0.05 = 3.45

dotProject = 2 x 0.3 + 2 x 0.3 + 3 x 0.1 + 2 x 0.1

+ 2 x 0.05 + 2 x 0.1 + 2 x 0.05 = 2.1

Odoo = 3 x 0.3 + 4 x 0.3 + 4 x 0.1 + 3 x 0.1 + 4 x

0.05 + 3 x 0.1 + 4 x 0.05 = 3.5

Score

OpenProject

dotProject

Odoo

TOTAL

3.45

2.1

3.5

Therefore, we obtained, that the best tool is Odoo,

with a score of 3.5. OpenProject has a very close score

of 3.45, and dotProject ranked last, with 2.1.

5 CONCLUSIONS AND FUTURE

WORK

Project management is a fundamental requirement in

any software competitive business. The use of project

management tools is a key to overcoming the

management complexity. However, there are many

project management tools available and the current

challenge resides in picking the right one. This work

aimed to compare three project management tools,

namely OpenProject, dotProject and Odoo, using the

OSSpal methodology. To perform this analysis, we

carried out several tasks, from the study of the

method, to its practical application, in order to obtain

results. The use and application of this methodology

allowed a broad and clear, weighted, scientifically

based view, demonstrating that the best tool is Odoo.

The perceived advantages were clearly evidenced,

allowing a better perception of the use of this tool in

project management.

Odoo has proved to be an easy-to-use tool,

centralizing deadlines, costs and teams, thus

performing good project management, which is vital

in a company, organizing information and automating

actions, providing good execution and monitoring of

projects, in the aspects that are most important. In

general, the other tools analysed were very good, but

some characteristics of Odoo differentiated it,

ensuring, based on the parameters, that companies

have a good tool at hand.

Although OpenProject and dotProject fulfill the

objectives that are proposed, we see that in the

categories “Software Technology Attributes” and

“Community and Adoption” Odoo proved to be more

capable.

As future work, we intend to include more

metrics in order to refine the evaluation, and compare

more tools, paying special attention to emerging

technologies.

REFERENCES

Abramova, V., Pires, F., & Bernardino, J. (2016). Open

Source vs Proprietary Project Management Tools. (Á.

Rocha, A. M. Correia, H. Adeli, L. P. Reis, & M.

Mendonça Teixeira, Eds.) New Advances in

Information Systems and Technologies, pp. 331-340.

Akita, F. (2015, 10 28). Answer to the Quora question

"What is the best software based in the PMBOK 5th

edition?". Retrieved 06 25, 2019, from Quora:

https://www.quora.com/What-is-the-best-software-

based-in-the-PMBOK-5th-edition/answer/Fabio-Akita

Boulanger, A. (2005). Open-source versus proprietary

software: Is one more reliable and secure than the

other? IBM Systems Journal, 44(2), 239-248.

Davis, D. (2004, 9 1). The PM Prescription. Retrieved 06

25, 2019, from Dr. Dobb's, web page:

http://www.drdobbs.com/the-pm-

prescription/184415189

dotProject main page. (2019). Retrieved 06 25, 2019, from

https://dotproject.net/

Ferreira, T., Pedrosa, I., & Bernardino, J. (2018).

Evaluating Open Source E-commerce Tools using

OSSpal Methodology. Proceedings of the 20th

International Conference on Enterprise Information

Systems - Volume 1: ICEIS, pp. 213-220.

doi:10.5220/0006790902130220

Gonçalves, ,. R., Kühlkamp, E., & Gresse von

Wangenheim, C. (2015). Enhancing DotProject to

Support Risk Management Aligned with PMBOK in

the Context of SMEs. International Journal of

Information Technology Project Management

(IJITPM) 6(2) .

Gonçalves, R. Q. (2018). Evolution of dotProject. Retrieved

06 25, 2019, from GQS - Software Quality Group,

WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies

416

home page: http://www.gqs.ufsc.br/evolution-of-

dotproject/

Lindenthal, B. (2019, 1 23). Step by step guide to

synchronize your Excel Sheet with OpenProject.

Retrieved 06 25, 2019, from OpenProject web site:

https://www.openproject.org/synchronize-excel-

openproject/

Marinheiro, A., Bernardino, J. (2015). Experimental

evaluation of open source business intelligence suites

using OpenBRR. IEEE Latin America Transactions

13(3), 810–817.

Odoo main page. (2019). Retrieved 06 25, 2019, from

https://www.odoo.com

OpenBRR. (2005). OpenBRR White paper. Retrieved 06

25, 2019, from

https://web.archive.org/web/20050803022846/http://w

ww.openbrr.org/docs/BRR_whitepaper_2005RFC1.pd

OpenProject main page. (2019). Retrieved 06 25, 2019,

from https://www.openproject.org/

OSSpal software list. (2019). Retrieved 06 25, 2019, from

OSSpal web page: http://osspal.org/

Pereira, A. K., Sousa, A. P., Santos, J. R., & Bernardino, J.

(2018). Open Source Data Mining Tools Evaluation

using OSSpal Methodology. International Conference

on Software Technologies (pp. 672-678). SciTePress.

PMI. (2017). A guide to the project management body of

knowledge (PMBOK guide) (6th ed.). Newtown Square,

Pennsylvania, United States of America: Project

Management Institute, Inc.

Wasserman, A. I., Guo, X., McMillian, B., Qian, K., Wei,

M.-Y., & Xu, Q. (2017). OSSpal: Finding and

Evaluating Open Source Software. In F. Balaguer, R.

Di Cosmo, A. Garrido, F. Kon, G. Robles, & S.

Zacchiroli (Eds.), Open Source Systems: Towards

Robust Practices (pp. 193-203). Springer International

Publishing.

Wheeler, D. A. (2015). Why Free-Libre/Open Source

Software (FLOSS)? Look at the Numbers! From

https://dwheeler.com/oss_fs_why.html

An Application of OSSpal for the Assessment of Open Source Project Management Tools

417