A SYSTEM ON WEB-BASED CONTINUOUS SOFTWARE
PROCESS ASSESSMENT (CONTINUOUS SPA)
Xian Chen and Paul Sorenson
Department of Computing Science, University of Alberta, Edmonton, AB, Canada
John Willson
Administrative Information Systems, University of Alberta, Edmonton, AB, Canada
Keywords: Continuous SPA, Web-based process assessment, Software process monitoring, Global management.
Abstract: Software process assessments are now recognized as important quality improvement activities in the
software industry. However, most assessment applications are generally regarded as infrequent, expensive
and disruptive for the workplace. Hence, it is advantageous to find alternative ways to the current status of
software processes and monitor the implementation of improvement activities. In this paper, we focus on
continuous process assessment and capability monitoring. A web-based prototype system is developed to
perform a practical study on continuous software process assessment in one process area: project
management. The study results show that features such as global management, well-defined responsibility
and visualization may help improve the efficiency and continuity of software process management.
1 BACKGROUND
Since the 1990's the software process community has
been growing in response to the increasing
importance of software in industry (Humphrey
1999). Many different standards have been created to
help control software quality and production. Up to
2005, the software process assessment (SPA)
approaches, such as CMMI (CMMI, 2000; 2002),
ISO 15504 (ISO/IEC, 1998; 2003) (also know as
SPICE), BOOTSTRAP (Kuvaja et al., 1994; 1999),
etc., achieved more than others (Rout, 2003).
Although assessments are useful, they are also
regarded as expensive and disruptive to the actual
software development work and often provide little
operational guidance to an improvement programme.
The Deming Cycle (Deming, 1986) shows that
selecting a well-defined approach can be a good start
to guide the software process improvement effort.
Based on the cycle, we are able to find alternative
ways to assess software processes continuously and
monitor the implementation of improvement
activities.
In this paper, we report on the development a
tool and an associated study of continuous software
process assessment. The paper is organized as
follows: section 2 reviews the related work on
PROFES; section 3 defines the study outline and the
web-based system that implements our study; section
4 describes an example project that uses the system
to help with the project management assessment and
section 5 summarizes the advantages and drawbacks
within the example. The final section gives short
conclusions and prospects for future work.
2 RELATED WORK
In 2000, the European project PROFES (Birk et al.,
1998) (PROduct-Focused improvement of
Embedded Software process) integrated process
assessment and the goal-driven software
measurement (GQM) (Basili et al. 1994) to enable
continuous assessment. The regular steps In
PROFES to apply continuous assessment can be
summarized as follows (Jävinen and Solingen, 1999)
(Jävinen, 2000):
1. Select processes based on standard process
areas (e.g. those defined in ISO-15504);
2. Define indicators for process existence
dimension and capability dimension;
3. Construct or update GQM and measurement
plans;
529
Chen X., Sorenson P. and Willson J. (2007).
A SYSTEM ON WEB-BASED CONTINUOUS SOFTWARE PROCESS ASSESSMENT (CONTINUOUS SPA).
In Proceedings of the Ninth International Conference on Enterprise Information Systems - ISAS, pages 529-532
DOI: 10.5220/0002367205290532
Copyright
c
 SciTePress

4. Collect data and analyze results. Software
tools are used for managing the measurement data
and producing graphs for analysis sessions.
Experiences in PROFES show that the benefits of
continuous assessment include (Jävinen et al. 1999):
 Improved visibility of software processes;
 Early detection of process deviations;
 Reduced cost of assessment.
3 WEB-BASED CONTINUOUS
PROCESS ASSESSMENT
(CONTINUOUS SPA)
3.1 Motivation
As indicated in ISO 15504(SPICE) (ISO/IEC, 1998;
2003), the main idea of the continuous assessment is
to provide an assessment tool that supports
collection of data (metrics) in the assessment of
process capability.
We are focusing on integrating the following
features into assessment instrument:
 Distributed Management: the accessibility of
the system should be distributed;
 Well-defined responsibility: user, role and
permission management should be well-
defined and organized in the system;
 Visualization: the high-level graphs and
visualized indicators should be used to help
the project participants.
3.2 Purpose
According to the instructive practice of PROFES, it
is practical to follow the software process
improvement cycle and define the continuous
process assessment into step-by-step activities. Our
study proceeded by the following steps:
1. Select processes based on ISO-15504 process
areas (in the study outlined in this paper, we focus
on the Project Management process area).
2. Define products in the selected processes.
3. Construct the measurement plans by
determining key process metrics related to the
products defined in step 2.
4. Collect and analyze data. Software tools are
used to assist in managing and analyzing the
collected data.
We developed a prototype system (Continuous
SPA) as the tool to perform these four steps.
3.3 System Architecture
Figure 1 shows the system architecture. The web
server is the main component of the application,
which consists of Servlet, JSP and a set of JAVA
classes (some commonly used components might
also be included in this part). The main Servlet/JSP
component is made up of three subcomponents:
Project Management, User/Role Management, and
Quality Management. The web server interacts with
the Database, processes the data, and creates the web
pages for the Web Client.
Figure 1: Overview of the System Architecture.
3.4 System Functionalities
Continuous SPA has the following functionalities
that help instrument the process assessment:
 construct and manage the work products in
project management;
 manage the quality measurements during the
product quality life-cycle;
 manage basic information in a project like
document management, role management,
user & team management, etc;
 integrate the rating function like the SPICE tool
or other assessment tools, and provide the
indicators for the quality and capability level
of the project management.
4 CASE STUDY: ASSESSING
PEOPLESOFT ASSET
MANAGEMENT
We use the PeopleSoft Asset Management project as
an example to show how to apply Continuous SPA
in assessing and monitoring the project management.
The project is contracted between Administrative
Information Systems (AIS) of University of Alberta
ICEIS 2007 - International Conference on Enterprise Information Systems
530

and a contracted developer (whose name we are not
allowed to specify here, abbr. CD in this paper).
The project’s goal is to implement the PeopleSoft
Asset Management module within the Financial
Services department to manage their building assets,
and we use the first phase of the project for our case
study. The study proceeded by three steps to map the
project properly into Continuous SPA:
1. Define the workflow of the project
products;
2. Define the roles for project management;
3. Map the metrics to the project products.
4.1 Define the Workflow of the Project
Products
According to the general AIS Methodology
Overview (University of Alberta, 2005), the
elements in a deliverable workflow can be
summarized as Project Charter and Statement of
Work (abbr. SOW) followed by a list of deliverables.
The following rules are applied to map the project to
the deliverable workflow:
 Use blocks to represent the deliverables and
milestones.
 Use arrows to represent the dependencies and
timelines of the deliverables;
 Use parallel arrows to represent the deliverables
proceeded simultaneously.
4.2 Define the Roles and Permissions
for Project Participants
In Asset Management, project deliverables are
tangible items that CD delivers to the University for
their review and approval.
The detailed descriptions and responsibilities of
the participants are given in SOW. A participant is
categorized into one of the two role groups: CD
groups and University groups. Each group has a set
of roles.
We defined these roles in Continuous SPA, then
granted the permissions (so far three types of
permissions: review, modify and rate) for the
deliverables defined in step 1 to these roles
according to SOW.
4.3 Map the Metrics to the Project
Products
We can define metrics according the AIS Project
Charter (University of Alberta, 2005) and relate the
metrics to deliverables as follows:
1. Select the subset of the measures from the
reference model;
2. Define and assign the measures to the related
products;
3. Update the values of the measures during the
project management.
In the study, we chose those service measures
that are easy to handle and defined the corresponding
metrics in the system. Then we defined the expected
values of the metrics. The service standard level can
be put into the metrics description for helping
assessors to give the correct values during the rating.
In general, a percentage rating system can be used as
the standard indicator values. Once the metrics
characteristics are defined, the process monitoring
can begin. Figure 2 shows a rating scenario for
SOW.
Figure 2: Project Map with Rating Report of SOW
(Colorized Sheet).
5 DISCUSSION
In the step-by-step study just outlined above, we
described the way Continuous SPA can be used to
manage, assess and monitor an ongoing project. The
example gave us positive appraisal on mapping
local, manual and periodical management process
into a global, automatic and continuous process. The
following useful features are helping us improve and
rethink our software process management approach:
 Globalization. The use of web-based
application allows process improvement to be
global, collaborative and shared.
 Security. By using the user/role/permission
subsystem, the tension between security and
globalization can be largely eliminated in our
system.
 Using Metrics. The use of metrics assists in the
development of a well-defined communication
channel between project requirement and
project assessment.
A SYSTEM ON WEB-BASED CONTINUOUS SOFTWARE PROCESS ASSESSMENT (CONTINUOUS SPA)
531

 Visualization. The system supports the
visualization of the product workflows as well
as quality measurement.
 Reusability. Each module in Continuous SPA
is responsible for one or more distinct aspects.
These modules were purposely developed into
independent components for further change or
reuse.
We came across some limitations in using
Continuous SPA. The three types of permissions
were not sufficient to define and manage all types of
responsibilities. We were also unable to map the AIS
project change control procedure because its
application cannot be predefined for a specific work
product. The prototype system needs to be enhanced
to model this dynamic behaviour.
6 CONCLUSION AND FUTURE
WORK
This paper defines and provides an initial evaluation
of a continuous software process assessment
prototype system that fills a gap between staged
process measurement and continuous process
improvement. Our approach integrated modules with
the following advantages:
 Web-based management that allows the
participants global access to and analysis of
up-to-date data;
 Well-defined responsibility for a participant
based on a user/role/permission management
system;
 Visualized indicators that represent
measurement results instantly and allow the
user easily identify process improvement
opportunities.
The case study shows promising results that
improve the efficiency and continuity of software
process management.
Our future work is to gather more experiences by
using the approach in new projects. We also want to
apply some of the important concepts and lessons
learned in this project to an examination of service
level agreement (SLA) tracking in systems built
using Web services and Service Oriented
Architecture (SOA).
REFERENCES
Humphrey, W.S., 1999. Competing in the Software Age.
Software Engineering Institute.
CMMI, 2000. CMMI-SE/SW, V1.0 Capability Maturity
Model® - Integrated for Systems
Engineering/Software Engineering, Version 1.0
Continuous Representation. Pittsburgh, PA. Software
Engineering Institute. CMU/SEI-2000-TR-019.
CMMI, 2002. CMMI-SW, V 1.1 Capability Maturity
Model® - Integrated for Systems
Engineering/Software Engineering, Version 1.1
Continuous Representation. Pittsburgh, PA. Software
Engineering Institute. CMU/SEI-2002-TR-028.
ISO/IEC, 1998. ISO/IEC-TR15504, Information
Technology - Software Process Assessment,
International Organisation for Standardisation (Ed.).
ISO/IEC, 2003. ISO/IEC-15504, Information Technology
- Process Assessment, International Organization for
Standardization (Ed.)
Kuvaja, P., Similä J., Krzanik, L., Bicego, A., Saukkonen,
S., and Koch, G., 1994. Software Process Assessment
& Improvement - The BOOTSTRAP Approach.
Oxford, UK. Blackwell Publishers
Kuvaja, Pasi, 1999. Bootstrap 3.0 - A SPICE Conformant
Software Process Assessment Methodology, Software
Quality Journal, 8,7-19
Rout, Terence P., 2003. ISO/IEC15504 - Evolution to an
International Standard, Software Process Improvement
and Practice, 8.27-40
Deming, W. E. 1986. Out of the Crisis: Quality,
Productivity and Competitive Position. Cambridge.
Mass.: MIT Center for Advanced Engineering Study.
Birk, A., Järvinen, J., Komi-Sirviö, S., Kuvaja, P., Oivo,
M. and Pfahl, D., 1998. “PROFES - A Product Driven
Process Improvement Methodology”. In the
Proceedings of Software Process Improvement 1998
(SPI´98), Monte Carlo, Monaco
Basili, V.R., Caldiera, G., and Rombach, H.D. 1994. “Goal
Question Metric Paradigm”. In J.J. Marciniak, ed.,
Encyclopedia of Software Engineering, vol. 1, pp.
528–532. John Wiley & Sons.
Jävinen, J., Solingen R. van, 1999. Establishing
Continuous Assessment Using Measurements. In
Proceedings of the 1st International Conference on
Product Focused Software Process Improvement
(PROFES'99). Oulu, Finland.
Järvinen, J., 2000. Measurement based Continuous
Assessment of Software Engineering Processes.
Järvinen, J., Hamann, D., Solingen R. van, 1999. On
Integrating Assessment and Measurement: Towards
Continuous Assessment of Software Engineering
Processes, In Proceedings of the 11th International
Conference on Software Engineering and Knowledge
Engineering (SEKE '99). Kaiserslautern, Germany,
June 17 -19, pp. 83 -87.
University of Alberta, 2005. AIS Methodology Overview.
University of Alberta, 2005. AIS Project Charter Template.
ICEIS 2007 - International Conference on Enterprise Information Systems
532