A Framework to Evaluate Software Developer’s Productivity
The VALORTIA Project
J. M. Sánchez-Begines
1
, F. J. Domínguez-Mayo
1
, M. J. Escalona
1
, M. Mejías
1
, N. Sánchez-Gómez
1
,
J. M. Bolívar
2
, E. Morillo
3
and P. Perejón
4
1
Web Engineering and Early Testing Research Group, (IWT2), Department of Computer Languages and Systems,
University of Seville, Seville, Spain
2
Optima Infinito, Madrid, Spain
3
Servinform, Seville, Spain
4
Proavan, Seville, Spain
Keywords: Software Quality, Methodologies, Model-Driven Engineering, Productivity, Software Development.
Abstract: Currently, there is a lack in companies developing software in relation to assessing their staff’s productivity
before executing software projects, with the aim of improving effectiveness and efficiency. QuEF (Quality
Evaluation Framework) is a framework that allows defining quality management tasks based on a model.
The main purpose of this framework is twofold: improve an entity’s continuous quality, and given a context,
decide between a set of entity’s instances on the most appropriate one. Thus, the aim of this paper is to
make this framework available to evaluate productivity of professionals along software development and
select the most appropriate experts to implement the suggested project. For this goal, Valortia platform,
capable of carrying out this task by following the QuEF framework guidelines, is designed. Valortia is a
platform to certify users' knowledge on a specific area and centralize all certification management in its
model by means of providing protocols and methods for a suitable management, improving efficiency and
effectiveness, reducing cost and ensuring continuous quality.
1 INTRODUCTION
Currently Software Quality management is a
concern in industry since a good management
ensures that a project, product or service meets the
expectations and interests of all placeholders.
Carrying out a first-class quality management
involves to steadily analyze and evaluate the present
state and to define policies, goals and plans leading
to improve and increase users and costumers’
satisfaction. This trend motivates the existence of
today's multiple lines of research centered on
improving efficiency and effectiveness in different
quality management processes and achieving
resources, processes and products with the best
quality guarantee.
An effective and efficient quality management
also demands to define a Quality Model that states
the required goals and objectives. A Quality Model
consists of a set of features and relationships among
them that constitute the basis for specifying quality
and the subsequent evaluation. Thus, defining a
Quality Model is not an easy, but a complicated task,
if there is no clear strategy defining the main goal
that interests us regarding a company’s quality
management.
However, selecting a Quality Model is essential,
because if not clearly established and designed, no
goal is reached and any project management is
difficult to succeed. That is why a strategy is
compulsory to analyze, evaluate and manage quality
models effectively and efficiently. In conclusion,
planning first a good strategy defining the design
basis of a Quality Model is needed to elaborate one
that meets the expectations.
Finally, continuous quality must be implemented
to develop quality management, as well as to learn
how to improve quality management processes
through their control and the results of improving
the Quality Model. Consequently, it is necessary to
define a framework that may allow managing
quality, ensure its continuous effective and efficient
160
Sánchez-Begínes J., Domínguez-Mayo F., Escalona M., Mejías M., Sánchez Gómez N., Bolívar J., Morillo E. and Perejón P..
A Framework to Evaluate Software Developer’s Productivity - The VALORTIA Project.
DOI: 10.5220/0005556701600167
In Proceedings of the 10th International Conference on Software Engineering and Applications (ICSOFT-EA-2015), pages 160-167
ISBN: 978-989-758-114-4
Copyright
c
2015 SCITEPRESS (Science and Technology Publications, Lda.)
improvement, and implement appropriate
management tools in such proposals.
The QuEF framework is found in these lines of
research to analyze and evaluate quality models. It
allows a study of quality models to choose the most
suitable alternative for particular resources,
processes and products.
Particularly, this article constitutes a resource: an
IT professional. It lays the groundwork that can
establish a framework to obtain a management
environment for a given entity, in an efficient and
effective quality framework where an entity may be
any type of resources, processes or products. This
work can be generalized to study and assess the
quality for a particular entity from two points of
view: the owner’s and the interested organizations’.
The best productivity of technicians who will
develop the project is listed as an important factor in
software quality. Therefore, candidates who will
participate in the project must be examined to
determine whether they are suitable enough to
perform it.
The productivity of technicians who will
participate in a project will be assessed before
tackling its development, in order to carry out the
project's goal and improve efficiency and
effectiveness. For this aim Valortia, a platform to
evaluate users’ skills and certify their knowledge on
a particular subject is presented. It perfectly fits the
QuEF framework as it takes into account Valortia's
two main points of view: offer (evaluate productivity
of professionals along software development) and
demand (select the most appropriate experts to
develop the suggested project)
Valortia has evolved into a powerful testing tool
following the examples of the most used Web tools
in this field, so as to identify whether the selected
technicians are possible candidates to carry out the
project,. In addition, Valortia uses the concept of
tags allowing building their models according to ISO
standards for people’s accreditation, which gives this
tool a very relevant value in the face of
organizations concerned with the certification of
their staff.
These standards are vital in organizations
because with them, they increase their value in
products and offers to be endorsed by recognized
guidelines and standards. Similarly, ensuring the
smooth running of staff evaluations constitutes the
main objective of the tool presented in this article.
Thanks to these rules, Valortia becomes a different
test platform than the typical tools, meaning a very
remarkable product for businesses.
After this introduction, this paper is structured as
follows: Section 2 summarizes the work context of
Valortia project and the related work. Then, Section
3 presents the QuEF framework and how it
associates with Valortia. Section 4 describes the
skills and attitudes that Valortia covers.
Subsequently, Section 5 analyzes Valortia platform
and finally, Section 6 states conclusions and future
work resulting from Valortia project.
2 WORK CONTEXT AND
RELATED WORK
This section introduces the real Valortia project,
which is a platform to evaluate users' skills and
certify their knowledge on a particular subject.
Productivity is a model that consists of two parts:
skills (professional knowledge and technical
competences in software development) and attitudes
(professional conduct or behavior in certain
situations).
This tool has been developed according to the
QuEF Framework guidelines, which combines
phases with the idea of labels. Following this
method, Valortia can carry out its goals in a simple,
fast and effective way.
Valortia aims to achieve a framework to raise the
discussion to the European e-Competence
Framework (e-CF) (European e-Competence, 2014)
that provides a clear guidance both for private and
public businesses and organizations that need to
make decisions on the candidates they are going to
evaluate. It is also useful to promote clearer
understanding of ICT organizations' skills,
professions and professional perspectives needed.
Specifically, it focuses on the necessary skills to:
Develop, maintain and manage ICT projects
and processes.
Exploit and use ICT in organizations.
Make decisions, develop strategies and
promote the use of ICT.
The recipients of this framework are:
ICT professionals and managers.
HR executives.
Educational Institutions and training centers.
Providers of certifications.
Market analysts and policy makers.
It is divided into 4 dimensions:
Dimension 1: 5 areas of expertise: PLAN -
BUILD - RUN - ENABLE - MANAGE.
Dimension 2: 40 ICT skills.
Dimension 3: 5 levels of professionalism.
Dimension 4: Examples of knowledge and
AFrameworktoEvaluateSoftwareDeveloper'sProductivity-TheVALORTIAProject
161
skills.
The previous framework of reference lays the
foundation to develop assessment tools that enable
IT professionals and businesses to have knowledge
on:
The current profile of each ICT technician.
Plan or goals of professional development.
Analysis of job profile target to fulfill.
Study of current knowledge or skills.
Examination of the gap related to the target
position to fill.
Investigation on the most valuable
certifications for profile target to complete.
Training and certification plan.
In addition, ISO/IEC 17024:2012 (ISO/IEC
17024, 2012) is crucial since the platform can create
certificates for companies depending on the their
projects contexts. It constitutes a specification for
managing organizations and institutions that apply
for accreditation and international validation in the
field of people's certification.
ISO/IEC 17024 responds to the need to establish
an internationally recognized scheme for the
certification of individuals who operate it together
with agencies. It applies to any discipline that tends
to demonstrate competence of individuals under
certification.
ISO / IEC 17024 requires that both the
organization that operates under the management of
people as well as certified individuals demonstrate
proficiency. Competition implies that people have
been tested and validated on defined requirements
for education, knowledge, experience and skills.
This standard provides the general requirements
for implementing, maintaining and certifying
people’s management authorization, describing the
processes to certify competence and keep in secret
the information obtained, considering the concerns
of those affected and the organization itself as
competent to carry out the validation.
A certification generally verifies the quality of a
product, an organism or an individual. In the last
case, it means that someone possesses a proficiency
level to work properly and adequately to provide the
expected services. In the field of Information and
Documentation, certification represents the set of
tests to obtain a certificate attesting professional
qualification at a given moment in someone’s career.
Certification ensures that a professional who
possesses certain levels of knowledge and skills can
perform a job in the best possible conditions. Apart
from academic considerations, or values, above all,
the degree of adaptation to the requirements of
professional practice and development prospects
must also be considered. Certification also offers
professionals a tool for assessing levels of
competition in the area, as well as clarifies and helps
identify the suitable profile of candidates for a job,
thereby providing greater transparency elements and
saving functioning in the labor market.
Many bodies responsible for this authorization
exist to support these certifications, such as:
ENAC, which is responsible for accrediting
organizations by assessing compliance with
UNE-ISO / IEC 17024: 2012 requirements. It
certifies professionals through their evaluation
in order to assess whether they meet the
requirements previously agreed.
AENOR (Spanish Association for
Standardisation and Certification), which is an
organization standing for the development of
standardization and certification (N + C) in
industrial and service sectors..
As related work, it can stand out the different
tools that today exist for conducting exams.
BlackBoard, WebCT and Moodle are examples of
these tools.
BlackBoard Testing tool is a kind of strategy for
assessing students’ performance, since it allows
integrating sets of items to one or more topics. This
tool scores some types of questions in a systematic
way and records some others manually.
WebCT has a tool to create test, self-evaluations
and surveys. The teacher first asks questions in a
database of questions. Then he/she elaborates tests
incorporating questions from the database. The
student finishes the exams and they can know their
scores and possible solutions. The teacher can view
the results: grades, student responses or statistics, for
example.
It also uses Quizzes/Surveys to create and handle
tests and questionnaires:
Exams are tests with assigned ratings. Both
students and teachers can access grades and
statistics.
Questionnaires are anonymous tests with no
grades assigned, but statistics. The answers to
the questionnaires are automatically graded
and the results are summarized.
Moodel platform has a wide variety of activities
that can be used during the students’ learning
process. Some can be used to publish materials;
others for students to perform work individually or
in groups (wikis, databases or workshops, among
others) and some others may be especially useful for
assessing or review students’ knowledge. The last
group copes with tasks and activities, especially
questionnaires.
ICSOFT-EA2015-10thInternationalConferenceonSoftwareEngineeringandApplications
162
Besides the possibility of creating exams easily with
high- quality results, Valortia hides in the structure a
very important factor. It is guided by the above
standards, so that it ensures quality guarantees that
the old tools do not provide.
Therefore, Valortia aims to consolidate as a tool
to accredit professionals with different certifications
offered and supported by the standards on which
they are based.
3 A FRAMEWORK TO
EVALUATE QUALITY
QuEF (Quality Evaluation Framework) is a
framework that allows defining quality management
tasks according to a model, so that all services the
software used can be generated automatically or
semi-automatically attending to the definition of that
model. It is based on the set of good practices known
as ITILv3 (Information Technology Infrastructure
Library) (ITIL, 2011).
QuEF was initially developed to analyze, assess
and improve quality of Model-Driven Web
Engineering (Escalona, 2004; Escalona, 2008;
Domínguez-Mayo, 2014; Vallecillo, 2007;
Schwinger, 2008) proposals, but it has now become
a generic framework for handling the quality of
resources, processes and products. It aims to plan,
control and improve those elements of an
organization that affect customers' satisfaction and
then achieve the result the organization expects.
Figure 1: Schematic QuEF framework supporting the set
of QuEF factory (Domínguez-Mayo, 2012).
As shown in Figure 1, QuEF Quality Model
lifecycle (Domínguez-Mayo, 2012) is organized into
the following phases:
Quality Model Strategy phase: It is
conceived as the heart of the Quality Model
lifecycle concept and its main objective states
that quality management should become
strategically active.
Quality Model Design phase: It deals with
the Quality Model design, processes, and
other aspects of the final design management
effort. Significantly, design in QuEF is
understood to encompass all relevant elements
to elaborate the Quality Model.
Quality Model Transition phase: It consists
in changing the Quality Model, without
influencing the Operation phase, which is
related to the Quality Model changes
management.
Quality Model Operation phase: It performs
the analysis, evaluation and plan of the
approaches continuous quality improvements.
In this phase, the Quality Model is used to
manage the quality of approaches.
Continual Improvement phase: It aims to
align and realign the Quality Model with the
properties to be covered and quality
characteristics to be agreed with the approach
development teams. The Quality Model can
change due to identified new trends or
technology changes.
As previously mentioned, the QuEF framework
revolves around a Quality Model, which was
composed of the following elements in the early
QuEF:
Model: It refers to the model to develop so as
to carry out various operations that are
performed in QuEF and to which all the
techniques and concepts defined in the
framework are applied.
Features: This is a set of properties of the
previous model.
Sub-features: It is a branch within the above
features to expand their level of detail.
Properties: They should indicate the
description of the characteristics and needs the
Quality Model must cover.
These properties can be assigned to previously
defined metrics. They indicate the type of measures
that properties are undergoing, which have a range
of values defined by the designer of the model who,
in turn, will make some judgments of the properties
according to users’ needs.
Furthermore, Quality Characteristics also refer to
the Quality Model. They point out the characteristics
AFrameworktoEvaluateSoftwareDeveloper'sProductivity-TheVALORTIAProject
163
derived from software products. Quality
Characteristics have a relationship with Features. It
is necessary to meet a number of requirements
defined by the Features to achieve one of these
characteristics.
All these concepts have been adapted according
to ISO/IEC 15939, which defines a measurement
process applicable to all disciplines of system and
software engineering and management.
Nowadays, the model has been transformed to
facilitate adaptability and flexibility in different
domains. Now, the Quality Model in QuEF is
represented by hand, a model of labels (tags model)
and items (this will replace Properties) that are
labeled through the model labels.
Figure 2: Tag model.
Another objective of the framework is the
automation of all processes currently developed to
support QuEF. It should provide tools that support
each of the phases defined in QuEF in relation to
quality management in order to transform the QuEF
framework into a business reality.
In addition, thanks to the definition of the
Quality Model, this knowledge can be shared among
those responsible for quality management in the
same context in order to:
Build quality control tools, application and
analysis of process control and inspection.
Use metrology and statistical methods to
diagnose and correct improper practices in
quality control.
Understand human factors and motivations.
Working with concepts and quality costs
techniques, having knowledge and ability to
develop and handle information management
systems.
Audit quality systems regarding identifying
and correcting deficiencies.
In addition, by changing many of the concepts,
tools have also evolved. This is the case of Valortia
tool; a powerful platform that adapts to the new
definition of the Quality Model previously
identified, allowing a quick and efficient study,
reliably backed up by quality standards results. The
purpose of the QuEF framework is to converge
towards quality continuous improvement through
tags, automated assessment and plans for monitoring
and improving quality automatically, in addition to
reduce effort, cost and time.
4 A MODEL TO EVALUATE
PRODUCTIVITY: SKILLS AND
ATTITUDES
In Valortia project, productivity has been defined as
a model that consists of two parts: skills (experts'
knowledge on software development together with
their technical skills and competences in that area)
and attitudes (conduct or behavior of technicians
when facing particular situations).
As skills concerns, the European competence
framework that defines the e-CF framework is taken
as reference. Valortia takes into account the
following different models of skills:
Application Development: Interpret
application design to develop an appropriate
application according to the customer’s
requirements and tend to adapt existing
solutions, by encoding, debugging testing,
documenting and communicating at every
stage of product development.
Component Integration: Integrate hardware,
software or system components into an
already existing or a new system complete
processes and procedures. Take into account
the compatibility of already existing and new
modules to ensure system integrity, system
interoperability and information security.
Testing: Build and execute systematic test
procedures for IT systems or customer
usability requirements to satisfy design
specifications.
Documentation Production: Elaborate
documents describing products, services,
components or applications that comply with
relevant documentation requirements by
selecting the appropriate style and
presentation materials in the media.
Problem Management: Identify and resolve
the root cause of incidents as well as take a
proactive approach to prevent and find out the
root cause of ICT problems. Deploy a
knowledge system based on the repetition of
ICSOFT-EA2015-10thInternationalConferenceonSoftwareEngineeringandApplications
164
common errors. Solve or escalate incidents
and optimize the system or component
performance.
Valortia takes into account the models of
attitudes that are listed below (Optima Infinito,
2014):
Global Vision: Have a complete integrated,
updated and always available perspective to
overview the existing commitments, and be
aware of the consequences of success or
failure to meet each of them.
Focus on results: Identify, in a very concrete
and specific way, the expected outcome of
each of the actions as well as the sequences of
steps leading to their achievement.
Analytical Thinking: Define the meaning of
“work” in a concrete way, transforming the
inputs received into specific and manageable
actions and results.
Decision Making: Decide systematically and
objectively what to do, based on the actual
circumstances and needs of the moment.
Effective Delegation: Systematically delegate
any action that can be performed by another
person and manage delegated actions
effectively.
Commitments Management: Keep an
integrated update and effective organization
system, including an updated inventory of all
the existing commitments and reviews to
dynamically renegotiate them, depending on
the actual circumstances and needs.
Personal effectiveness: Achieve results in an
optimal way in terms of effectiveness,
efficiency and quality. Effectiveness: get the
most relevant results first. Efficiency: use
resources optimally. Quality: maximize the
generation of added value.
Effective Communication: Communicate in
a direct, concise, specific, proactive and
assertive way, giving and asking for feedback,
to ensure that the information provided is fully
understood by all parties.
5 THE VALORTIA PROJECT
PLATFORM
Valortia is a platform for evaluating users' skills and
certifying their knowledge on a particular subject. It
can be used from two points of view: offer (evaluate
productivity of professionals along software
development) and demand (select the most
appropriate experts to develop the suggested
project).
On the one hand, if it is aimed to design a model
that could meet customers' needs and expectations,
the framework will allow analyzing, evaluating,
controlling and improving the model, so that these
needs could be fulfilled. Valortia helps award
certifications based on tags in order to know users’
capabilities. They enable assessing and analyzing
their knowledge on the subject.
Figure 3: Certification creation form.
These certifications will be created by
organizations that intend to find the best ways to
handle the resources, processes and products being
conducted. The creation form of these certifications
is very simple and intuitive. Note that the label
definition part is relevant as a distinctive feature to
each of the questions (feature) in the certification
(model) that can be assigned. These labels appoint a
weight according to the importance given to the
project and thus, they determine the best candidates
for that specific project.
This process is possible thanks to the concepts
and techniques the QuEF framework establishes.
Figure 4: Labels and weight in the certification.
AFrameworktoEvaluateSoftwareDeveloper'sProductivity-TheVALORTIAProject
165
On the other hand, deciding on the most appropriate
model in terms of personal interests means to be on
the side of customers, users and recipients of model
instances, thus the working framework may allow
comparing model instances to achieve the key aim.
As Figure 2 shows, Valortia platform offers values
concerning the productivity of professionals.
Customers can perform the certifications that fit the
context in which they will assess productivity.
Valortia offers a lot of facilities to make
certifications exams improve usability of tool and
respect availability of customers.
Figure 5: Evaluations made by a user.
Users may sit certification exams that are posted
on the application. Besides, an organizer may invite
a user to take a certification examination. In this
way, companies or organizations may send this
invitation to the candidates who will carry out the
project.
In accordance with the exams creation models of
the aforementioned applications, Valortia stores a
variety of questions types to create certifications.
These questions or exercises may be of the
following nature:
Free Text
Unique Selection
Multiple Choice
Questions may also have different levels of
difficulty and an established response type (numeric
or short/long text, for instance).
Finally, it is worth appreciating that different
responses, in cases where there is more than a true or
incomplete answer, may share scores.
Users will be doing these certifications
gradually, provided that they are within the
maximum set for the completion of the certification.
Figure 6: Multiple choice questions.
6 CONCLUSIONS
In conclusion, this paper shows that quality software
can be developed thanks to the guidelines based on
quality standards provided by the QuEF framework
for the analysis and evaluation of models. This will
allow studying and assessing them and it will be
friendly for organizations to use it, keeping quality
of results.
The concept of productivity has multiple
meanings in literature. In the current project, it
primarily focuses on all those attitudes and
competences (or skills) that ICT professionals
achieve. It also defines a model that includes the
following features: skills and attitudes
After a detailed analysis of the literature, we can
conclude that:
Productivity can and should be measured. In
the light of the success criteria of the project,
it must creatively seek measures to assess
progress in a systematic and regular way.
Productivity rarely can be compared to
external projects, unless the same indicators
and metrics are used.
Normally productivity can improve, but it
does not mean that all factors can. These
topics require not only a lot of experience to
provide an overview of the circumstances that
shape the project, but also a high dose of
inspiration to hit effectively with appropriate
and timely action.
Furthermore, it is very important to consider both
the rules in UNE-EN ISO / IEC 17024 and European
e-Competence Framework to certify people. These
standards have been the guidelines for the
development of Valortia. In consequence, this tool
can ensure quality assurance of the results obtained.
Besides, Valortia has methods and templates
available to create a Quality Model based on labels.
ICSOFT-EA2015-10thInternationalConferenceonSoftwareEngineeringandApplications
166
These labels are supported by the QuEF framework,
which, by assigning a weight, can be used for
analysis and evaluation in a quick and efficient way.
It is worth pointing out that Valortia creates
certifications based on the aforementioned skills and
ISO 17024. Thus, it ensures good practice guidelines
to evaluate customers. Finally, the QuEF framework
is starting to be applied in industry: THOT,
VALORTIA, ADAPT and CalipsoNEO are
examples of this practice.
As future line of research, there is still work with
THOT project, in order to make a decision on which
documentation tool and features are better.
Additionally, ERIS G3 project uses the QuEF
framework to compare organizations and decide on
the most suitable one to develop the project.
ACKNOWLEDGEMENTS
This research has been supported by the MeGUS
project (TIN2013-46928-C3-3-R) of the Spanish
Ministry of Science and Innovation.
REFERENCES
Domínguez-Mayo, F. J., Escalona, M. J., Mejías, M.,
Ross, M., Staples, G., 2014, "Towards a
Homogeneous Characterization of the Model-Driven
Web Development Methodologies". Journal of Web
Engineering, 13(1-2), 129-159.
Domínguez-Mayo, F. J., Escalona, M. J., Mejías, M.,
Ross, M., Staples, G., 2012, A quality management
based on the Quality Model life cycle, Computer
Standards & Interfaces, Volume 34, Issue 4, pp. 396-
412, ISSN 0920-5489.
Domínguez-Mayo, F. J., Escalona, M. J., Mejías, M.,
Ross, M., Staples, G., 2012, Quality evaluation for
Model-Driven Web Engineering methodologies,
Information and Software Technology, Volume 54,
Issue 11, pp. 1265-1282, ISSN 0950-5849.
Escalona, M. J., Aragón, G., 2008, “NDT. A Model-
Driven Approach for Web Requirements”. IEEE
Transactions on Software Engineering, Vol. 34, No. 3,
pp. 377-390.
Escalona, M. J., Koch, N., 2004, “Requirements
Engineering for Web Applications – A comparative
study”. Journal of Web Engineering. Vol. 2, No. 3, pp.
193-212.
ITIL Official site, 2011, http://www.itil-officialsite.com/
ISO/IEC 17024, 2012, Conformity assessment--General
requirements for bodies operating certification of
person, 2012, http://www.iso.org/iso/home/store/
catalogue_tc/catalogue_detail.htm?csnumber=52993.
Vallecillo, A., Koch, N., Cachero, C., Comai, S.,
Fraternali, P., Garrigós, I., Gómez, J., Kappel, G.,
Knapp, A., Matera, M., Meliá, S., Moreno, N., Pröll,
B., Reiter, T., Retschitzegger, W., Rivera, J. E.,
Schwinger, W., Wimmer, M., Zhang, G., 2007,
“MDWEnet: A Practical Approach to Achieving
Interoperability of Model-Driven Web Engineering
Methods”, Proc. Third Int’l Workshop Model-Driven
Web Eng., pp. 246-254.
European e-Competence Framework, http://profiletool.
ecompetences.eu/, Retrieved November 2014.
Schwinger, W., Retschitzegger, W., Schauerhuber, A.,
Kappel, G., Wimmer, M., Pröll, B., Cachero C.,
Castro, Casteleyn, S., De Troyer, O., Fraternali, P.,
Garrigos, I., Garzotto, F., Ginige, A., Houben, G-J.,
Koch, N., Moreno, N., Pastor, O., Paolini, P.,
Pelechano V., Ferragud, Rossi, G., Schwabe, D., Tisi,
M., Vallecillo, A., van der Sluijs. Zhang, G., 2008, “A
survey on Web modeling approaches for ubiquitous
Web applications”. International Journal of Web
Information Systems Vol. 4 No. 3, pp. 234-305.
Optima Infinito, http://www.optimainfinito.com,
Retrieved November 2014.
AFrameworktoEvaluateSoftwareDeveloper'sProductivity-TheVALORTIAProject
167
