An Application of OSSpal for the Assessment of Open Source Project
Management Tools
Hugo Carvalho de Paula
1
and Jorge Bernardino
1,2 a
1
Polytechnic of Coimbra, ISEC, Rua Pedro Nunes, Quinta da Nora, 3030-199 Coimbra, Portugal
2
CISUC - Centre of Informatics and Systems of University of Coimbra, Pinhal de Marrocos, 3030-290 Coimbra, Portugal
Keywords: Software Assessment, Project Management.
Abstract: Projects are a necessity within any competitive business, and as the execution of complex projects becomes
the norm, so grows the need for advances in project management. The use of project management tools is key
towards taming said complexity. There are many such tools available; the current challenge resides in picking
the right one. In this paper, we evaluate three different tools - OpenProject, dotProject, and Odoo - using the
OSSpal methodology.
1 INTRODUCTION
Project management (PM) is the application of
knowledge, skills, tools, and techniques to project
activities to meet the project requirements (Project
Management Institute, 2017, p. 10). One of the
activities in a project is PM itself; therefore, a tool
must be applied to PM.
A natural question is how to properly determine
which tool to use. First, we recognize that PM is a
complex task, and that various kinds of software are
well-suited for specific subtasks in PM. We would
like to evaluate, then, general software; that is, tools
that assist PM throughout the whole project lifecycle.
Second, we prefer open-source software. The debate
between open-source and proprietary is beyond this
paper’s scope, but interested readers might want to
read papers like (Wheeler, 2015) or (Boulanger,
2005) for more general research, and (Abramova,
Pires, & Bernardino, 2016) for research that’s also
related to PM tools.
To answer the question, we turn to the field of
Open Source Software Assessment. It is a field that
has sprung up in response to the several challenges
brought by the increasing adoption of Open Source in
various enterprise projects. Practitioners in this field
have developed Open Source Software Assessment
Methodologies (OSSAMs, or just OSSAM in the
a
https://orcid.org/0000-0001-9660-2011
singular), well-organized collections of methods,
suitable for the complete task of assessment.
We apply a well-known OSSAM, the OSSpal
methodology (Wasserman, et al., 2017). The OSSpal
methodology is the successor project to the Business
Readiness Rating (BRR) (OpenBRR, 2005). OSSpal
methodology combines quantitative and qualitative
measures for evaluating software in several
categories, resulting in a quantitative value that
allows the comparison between tools (Wasserman, et
al., 2017).
In the OSSpal website (OSSpal software list,
2019), one can find several evaluations published. In
addition to those, we cite (Pereira, Sousa, Santos, &
Bernardino, 2018) and (Ferreira, Pedrosa, &
Bernardino, 2018) as related work, as they use
OSSpal to evaluate software in areas such as Data
Mining and e-Commerce.
We apply OSSpal to assess the area of project
management tools, focusing on three different tools
for PM: OpenProject (OpenProject main page, 2019),
dotProject (dotProject main page, 2019) and Odoo
(Odoo main page, 2019). This paper presents the
results of this evaluation.
The paper is organized as follows. Section 2
presents the three PM tools that will be evaluated.
Section 3 describes the OSSpal methodology. Section
4 presents the results of the evaluation. Finally,
section 5 presents the conclusions.
Carvalho de Paula, H. and Bernardino, J.
An Application of OSSpal for the Assessment of Open Source Project Management Tools.
DOI: 10.5220/0008412404110417
In Proceedings of the 15th International Conference on Web Information Systems and Technologies (WEBIST 2019), pages 411-417
ISBN: 978-989-758-386-5
Copyright
c
2019 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
411
2 OPEN SOURCE PROJECT
MANAGEMENT TOOLS
Project management tools assist the manager in
estimating, controlling, and tracking projects. Such
tools can have a profound impact on the quality of
project management and, consequently, on the quality
of the project itself. These tools can provide several
features that assist the manager with vital project
decisions, such as reporting, scheduling control, and
automating some day-to-day activities.
Even within the open-source and general tools,
there are still many choices for the practitioner. Thus,
we only present here the specific software that was
studied in detail: OpenProject, dotProject and Odoo.
2.1 OpenProject
OpenProject is a web-based project management
system for location-independent team collaboration.
It is an Open Source software registered under the
GNU General Public Version 3 License and is
continuously developed by an open source
community.
Some of its features are:
Milestones management;
Issue and bug tracking;
Wiki;
Document management;
Forum.
Time tracking;
Project newsfeed.
OpenProject supports both traditional and agile
processes. Its agile support includes the usage of
taskboards to break releases into individual tasks and
perform status updates with “drag’n’drop” actions. In
Figure 1, we can see a screenshot of OpenProject in
action.
Figure 1: OpenProject interface.
2.2 dotProject
dotProject is an Open Source project management
tool distributed under the GNU General Public
Version 2 License. It is a web tool, written in PHP,
that uses a MySQL database. The goal of dotProject
as a tool is to unify the important information of the
project, presenting an overview of the tasks and
responsibilities.
There is no business behind the project, it is
managed, maintained, and developed by a group of
volunteers and by the users themselves who
furthermore provide software support.
The current version of the software is available on
GitHub, with the documentation and guidelines on
the proper use of the resources still in the process of
being moved to a newer page. Figure 2 presents the
interface of the tool in action.
Figure 2: dotProject interface.
2.3 Odoo
Odoo is an integrated ERP system, whose main
characteristics are: it is an open source system; it is
cross-platform, since through any web browser you
can access its interface; it is easy to handle; it interacts
with other applications, with PDF viewing, import
and export of Microsoft Office documents, and also
compatibility, for example, with Google Maps.
Odoo is not only a simple management platform,
it encompasses sales and project management in one
tool, and enriches it with a variety of functions, such
as MRP, Point of Sale, and e-commerce applications,
in order to provide a universal solution that can help
companies manage all kinds of operations.
Figure 3 showcases Odoo, as it is used for project
management.
Figure 3: Odoo interface.
WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies
412
3 OSSpal METHODOLOGY
Based on the knowledge acquired through the study
of the tools previously explained, we now turn to the
OSSpal methodology explanation.
OSSpal has emerged as a successor to Business
Readiness Rating (OpenBRR), to provide a reliable
and unbiased source for evaluating open source
software (Marinheiro et al., 2015). The goal is to be
an open, comprehensive and standard evaluation
model that is reliable, widely used and adjustable.
OSSpal combines quantitative and qualitative
assessment measures that allow to identify which
software has the best score. It can help companies,
government agencies and other organizations find
high quality open source software (Wasserman, et al.,
2017).
The OSSpal methodology works by scoring
software across seven different categories, and then
computing a total score for each software, based on
the individual category scores and previously
determined weights.
The categories of OSSpal methodology are the
following:
Functionality: How well will the software meet
the average user’s requirements?
Operational Software Characteristics: How
secure is the software? How well does the
software perform? How well does the software
scale to a large environment? How good is the
UI? How easy to use is the software for end-
users? How easy is the software to install,
configure, deploy, and maintain?
Support and Service: How well is the software
component supported? Is there commercial
and/or community support? Are there people
and organizations that can provide training and
consulting services?
Documentation: Is there adequate tutorial and
reference documentation for the software?
Software Technology Attributes: How well is
the software architected? How modular,
portable, flexible, extensible, open, and easy to
integrate is it? Are the design, the code, and the
tests of high quality? How complete and error-
free are they?
Community and Adoption: How well is the
component adopted by community, market,
and industry? How active and lively is the
community for the software?
Development Process: What is the level of the
professionalism of the development process
and of the project organization as a whole?
The OSSpal uses a very similar calculation as the
BRR. We can exemplify the OSSpal methodology
calculation with an image from that method (see
Figure 4):
Figure 4: BRR score calculation (OpenBRR, 2005).
This methodology is composed of four phases:
1. First Phase: it is necessary to identify
a software component list to be analyzed,
to measure each component in relation to
the evaluation criteria and removing
from the analysis any software
component that does not satisfy the user
requirements;
2. Second Phase: it should be attributed
weights for the categories and for the
measures:
a. Assign a percentage of
importance to each category,
totaling 100%;
b. For each measure within a
category, it is necessary to rank
the measure in accordance with
its importance;
c. Foreach measure within a
category assign the importance
by percentage, totaling all the
measures 100% of the category.
3. Third Phase: gather data for each
measure used in each category and
calculate its weighting in a range
between 1 to 5 (1 - Unacceptable; 2 -
Poor; 3 Acceptable; 4 -Very Good;
and, 5 - Excellent);
4. Fourth Phase: the qualification of the
category and the weighting factors
should be used to calculate the OSSpal
final score.
The category ‘Functionality’ is calculated
differently from the others. In this category is
An Application of OSSpal for the Assessment of Open Source Project Management Tools
413
intended to analyze and evaluate the characteristics
which the tools have or should have. The method to
assess this category is as follows:
A. Set down the characteristics to
analyze, scoring them from 1 to 3 (less
important to very important);
B. Classify the characteristics in a
cumulative sum (from 1 to 3);
C. Standardize the prior result to a scale
from 1 to 5.
Therefore, the Functionality category will have
the following scale:
Under 65%, Score = 1 (Unacceptable)
65% -80%, Score = 2 (Poor)
80% -90%, Score = 3 (Acceptable)
90% -96%, Score = 4 (Good)
Over 96%, Score = 5 (Excellent).
4 EVALUATION
To follow up and start the evaluation following the
OSSpal methodology, we carried out research,
installed the tools, and proceeded to the basic use,
from a user's perspective. From this process, we
obtain several results, to be detailed in the remainder
of this section.
For the first phase, after the initial research, we
separated the tools we found into three groups, based
on the following requirements:
Good integration with external software. First,
we determine the validity of this requirement
from (Akita, 2015). Then we decided to use
Microsoft Excel as a baseline, given its
ubiquity in the corporate world. Based on this,
the software with the best integration was
OpenProject, due to its custom synchronization
feature (Lindenthal, 2019);
Good integration with internal software. This
requirement appears as a counterpart to the
first, in such a way that we represent all the
tools that stand out for the good integration.
Odoo stood out among the others, as it is
actually an entire ERP, which happens to
include a project management tool.
PMBOK (PMI, 2017) compliance. The
PMBOK is one of the leading guides in the
world of project management, so it is only
natural to require software that incorporates its
lessons directly. We found dotProject as the
highlight here, due to the research found in
(Gonçalves R. Q., 2018).
The result of the first phase is, therefore, the
three tools that we will evaluate in more detail;
OpenProject, dotProject, and Odoo.
As part of the result corresponding to the second
phase of the methodology, we list in Table 1 the
weights of the criteria to be evaluated.
Table 1: Weights of each OSSpal evaluation category.
Category
Weight
Functionality
30%
Operational Software
Characteristics
30%
Software Technology
Attributes
10%
Documentation
10%
Community and Adoption
5%
Support and Service
10%
Development Process
5%
Total
100%
This determination of weights primarily reflects
the purpose of the evaluation, which is the use of the
tool in the business context.
The "Functionality" and "Operational Software
Characteristics" categories, which are of major
importance, since they provide the greatest support to
information technology users and professionals of
areas related to PM, have a 30% weight.
The "Software Technology Attributes" category
was weighed with a focus on architecture and design,
which rendered a weight of 10%.
For the "Documentation" category, we noticed
many tutorials and reference documentation for all
the software we analyzed. Therefore, we attribute
10% as a weight.
In the "Support and Service" category, we have a
weight of 10% because there are a large number of
people and organizations that can offer training and
consulting services.
At the lower end of the weights table, the
categories "Community and Adoption" and
"Development Processes" were assigned equal
weights of 5%, since it is very difficult for users to
analyse the level of professionalism of the
development process and organization of the project
as a whole, and since the typical enterprise user has
less of a need for a community.
The other part of the second phase result is the
determination of metrics for the categories, and the
associated weights. In this work, we did not consider
necessary to have any metrics, other than the special
case of the "Functionality" category. The next step,
therefore, is the determination of the specific
functionalities that interest the project management,
together with their weights. Such functionalities have
WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies
414
been derived from the reading of (Davis, 2004), and
are listed in Table 2.
Table 2: Weights for the characteristics of the functionality
category.
Characteristic
Weight
Task Scheduling
3
Resource Management
3
Collaboration
2
Time Tracking
3
Estimating
2
Risk
Assessment/Management
1
Issue/Change Management
2
Reporting/Charts
2
File Attachments
1
E-mail Notification
1
Process/Methodology
2
Portfolio Management
1
For the third phase, since the only metric used was
that of the functionalities, we had to collect
information in order to determine two sets of values:
of that metric, and of the other categories directly. We
now present first the results of tool analysis with
respect to the metric in Table 3.
Table 3: Score of each specific functionality.
Characteristics
Open
Project
dot
Project
Odoo
Task Scheduling
3
3
3
Resource
Management
3
3
3
Collaboration
2
2
2
Time Tracking
3
3
3
Estimating
2
2
2
Risk Assessment/
Management
0
1
0
Issue/Change
Management
0
0
2
Reporting/Charts
2
2
2
File Attachments
1
1
1
E-mail Notification
1
1
1
Process/Methodology
2
0
0
Portfolio
Management
0
0
0
Cumulative Sum
19
18
19
Percentage compared
to maximum (23)
82,6%
78,2%
82,6%
Normalization to 1-5
scale
3
2
3
As we can observe, the scores are mostly the
same, although dotProject’s is a bit lower. Most
features are available on all tools, except the
following:
Risk Assessment / Management: Only
dotProject has this functionality, through an
additional module (Gonçalves, Kühlkamp, &
Gresse von Wangenheim, 2015)
Issue / Change Management: This feature is
only available on Odoo, through a paid
application.
Process / Methodology: Only OpenProject
allows the choice of management in a
traditional or agile way.
Portfolio Management: None of the tools have
good portfolio management.
From the above data, through the calculation
indicated by the methodology, we have the value
assigned to the category "Functionality" for each tool.
Below, we present Table 4, with the values assigned
to that and the other categories.
Table 4: OSSpal score by category.
Score
OpenProject
dotProject
Odoo
3
2
3
4
2
4
3
3
4
4
2
3
3
2
4
3
2
3
4
2
4
The determination of each value in this table was
based on the following information:
Functionality: Determined from the
calculation described in at the end of
section 3.
Operational Software Characteristics:
Both OpenProject and Odoo meet
nonfunctional requirements well, but
dotProject has issues with usability,
installation, and a few others.
Software Technology Attributes: Odoo
stands out as an ERP.
Documentation: OpenProject has the
highest score because we observe both
quality and quantity of documentation.
dotProject appears with lower score; this
is mostly due to the age of the tool, but
what little there is has quality.
An Application of OSSpal for the Assessment of Open Source Project Management Tools
415
Community and Adoption: Odoo is the
most popular, followed by OpenProject,
and finally dotProject.
Support and Service: Both Odoo and
OpenProject have corporations that
provide support, but they are small.
dotProject still has some support, but it's
not much.
Development Process: dotProject is
currently developed by few people,
without much organization. The other
tools are products of a company, so it has
a high value in this criterion.
Following the methodology, we then proceed to
phase 4, where we calculate the final score of each
tool, applying the weights previously defined.
OpenProject = 3 x 0.3 + 4 x 0.3 + 3 x 0.1 + 4 x 0.1
+ 3 x 0.05 + 3 x 0.1 + 4 x 0.05 = 3.45
dotProject = 2 x 0.3 + 2 x 0.3 + 3 x 0.1 + 2 x 0.1
+ 2 x 0.05 + 2 x 0.1 + 2 x 0.05 = 2.1
Odoo = 3 x 0.3 + 4 x 0.3 + 4 x 0.1 + 3 x 0.1 + 4 x
0.05 + 3 x 0.1 + 4 x 0.05 = 3.5
Score
OpenProject
dotProject
Odoo
TOTAL
3.45
2.1
3.5
Therefore, we obtained, that the best tool is Odoo,
with a score of 3.5. OpenProject has a very close score
of 3.45, and dotProject ranked last, with 2.1.
5 CONCLUSIONS AND FUTURE
WORK
Project management is a fundamental requirement in
any software competitive business. The use of project
management tools is a key to overcoming the
management complexity. However, there are many
project management tools available and the current
challenge resides in picking the right one. This work
aimed to compare three project management tools,
namely OpenProject, dotProject and Odoo, using the
OSSpal methodology. To perform this analysis, we
carried out several tasks, from the study of the
method, to its practical application, in order to obtain
results. The use and application of this methodology
allowed a broad and clear, weighted, scientifically
based view, demonstrating that the best tool is Odoo.
The perceived advantages were clearly evidenced,
allowing a better perception of the use of this tool in
project management.
Odoo has proved to be an easy-to-use tool,
centralizing deadlines, costs and teams, thus
performing good project management, which is vital
in a company, organizing information and automating
actions, providing good execution and monitoring of
projects, in the aspects that are most important. In
general, the other tools analysed were very good, but
some characteristics of Odoo differentiated it,
ensuring, based on the parameters, that companies
have a good tool at hand.
Although OpenProject and dotProject fulfill the
objectives that are proposed, we see that in the
categories Software Technology Attributes and
Community and Adoption Odoo proved to be more
capable.
As future work, we intend to include more
metrics in order to refine the evaluation, and compare
more tools, paying special attention to emerging
technologies.
REFERENCES
Abramova, V., Pires, F., & Bernardino, J. (2016). Open
Source vs Proprietary Project Management Tools. (Á.
Rocha, A. M. Correia, H. Adeli, L. P. Reis, & M.
Mendonça Teixeira, Eds.) New Advances in
Information Systems and Technologies, pp. 331-340.
Akita, F. (2015, 10 28). Answer to the Quora question
"What is the best software based in the PMBOK 5th
edition?". Retrieved 06 25, 2019, from Quora:
https://www.quora.com/What-is-the-best-software-
based-in-the-PMBOK-5th-edition/answer/Fabio-Akita
Boulanger, A. (2005). Open-source versus proprietary
software: Is one more reliable and secure than the
other? IBM Systems Journal, 44(2), 239-248.
Davis, D. (2004, 9 1). The PM Prescription. Retrieved 06
25, 2019, from Dr. Dobb's, web page:
http://www.drdobbs.com/the-pm-
prescription/184415189
dotProject main page. (2019). Retrieved 06 25, 2019, from
https://dotproject.net/
Ferreira, T., Pedrosa, I., & Bernardino, J. (2018).
Evaluating Open Source E-commerce Tools using
OSSpal Methodology. Proceedings of the 20th
International Conference on Enterprise Information
Systems - Volume 1: ICEIS, pp. 213-220.
doi:10.5220/0006790902130220
Gonçalves, ,. R., Kühlkamp, E., & Gresse von
Wangenheim, C. (2015). Enhancing DotProject to
Support Risk Management Aligned with PMBOK in
the Context of SMEs. International Journal of
Information Technology Project Management
(IJITPM) 6(2) .
Gonçalves, R. Q. (2018). Evolution of dotProject. Retrieved
06 25, 2019, from GQS - Software Quality Group,
WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies
416
home page: http://www.gqs.ufsc.br/evolution-of-
dotproject/
Lindenthal, B. (2019, 1 23). Step by step guide to
synchronize your Excel Sheet with OpenProject.
Retrieved 06 25, 2019, from OpenProject web site:
https://www.openproject.org/synchronize-excel-
openproject/
Marinheiro, A., Bernardino, J. (2015). Experimental
evaluation of open source business intelligence suites
using OpenBRR. IEEE Latin America Transactions
13(3), 810817.
Odoo main page. (2019). Retrieved 06 25, 2019, from
https://www.odoo.com
OpenBRR. (2005). OpenBRR White paper. Retrieved 06
25, 2019, from
https://web.archive.org/web/20050803022846/http://w
ww.openbrr.org/docs/BRR_whitepaper_2005RFC1.pd
f
OpenProject main page. (2019). Retrieved 06 25, 2019,
from https://www.openproject.org/
OSSpal software list. (2019). Retrieved 06 25, 2019, from
OSSpal web page: http://osspal.org/
Pereira, A. K., Sousa, A. P., Santos, J. R., & Bernardino, J.
(2018). Open Source Data Mining Tools Evaluation
using OSSpal Methodology. International Conference
on Software Technologies (pp. 672-678). SciTePress.
PMI. (2017). A guide to the project management body of
knowledge (PMBOK guide) (6th ed.). Newtown Square,
Pennsylvania, United States of America: Project
Management Institute, Inc.
Wasserman, A. I., Guo, X., McMillian, B., Qian, K., Wei,
M.-Y., & Xu, Q. (2017). OSSpal: Finding and
Evaluating Open Source Software. In F. Balaguer, R.
Di Cosmo, A. Garrido, F. Kon, G. Robles, & S.
Zacchiroli (Eds.), Open Source Systems: Towards
Robust Practices (pp. 193-203). Springer International
Publishing.
Wheeler, D. A. (2015). Why Free-Libre/Open Source
Software (FLOSS)? Look at the Numbers! From
https://dwheeler.com/oss_fs_why.html
An Application of OSSpal for the Assessment of Open Source Project Management Tools
417