Software Engineering Standards and Guides for Very Small Entities

Implementation in Two Start-ups

Claude Y. Laporte

, Rory V. O’Connor

and Luis Hernán García Paucar

École de technologie supérieure, Montréal, Canada

School of Computing, Dublin City University, Dublin, Ireland

Universidad Peruana de Ciencias Aplicadas, Lima, Peru

Keywords: Very Small Entities, ISO Standards, ISO/IEC 29110, Certification, VSE.

Abstract: Very small entities, enterprises, organizations, projects or departments with up to 25 people, are very

important to the worldwide economy. However it has ben established that such entities often do not utilize

existing standards and frameworks. To address the needs of Very Small Entities (VSEs), a set of

international standards and guides known as ISO/IEC 29110 has been developed. In this paper we present

the results of early trials of this standard in two IT start-ups VSEs. A Peruvian VSE was recently audited

and issued an ISO/IEC 29110 certificate of conformity.

1 INTRODUCTION

For many small and very small software companies,

implementing controls and structures to properly

manage their software development activity is a

major challenge. Administering software

development in this way is usually achieved through

the introduction of a software process. All software

companies are not the same and vary according to

factors including size, market sector, time in

business, management style, product range and

geographical location. For example, a software

company operating in India may have a completely

different set of operational problems when compared

to a software company in Canada, Mexico or

Ireland. Even within a single geographical area such

as Ireland, the range of operational issues faced by a

small local Irish-owned firm can be radically

different to those affecting a multinational

subsidiary. The fact that all companies are not the

same raises important questions for those who

develop software process and process improvement

models. To be widely adopted by the software

industry, any process or process improvement model

should be capable of handling the differences in the

operational contexts of the companies making up

that industry. But process improvement models,

though highly publicized and marketed, are far from

being extensively deployed and their influence in the

software industry therefore remains more at a

theoretical than practical level.

Software development small and very small have

the challenge of handling multiple small-scale, fast-

moving projects allowing little room for unwieldy

management processes, but still requiring an

efficient and straightforward monitoring process

(Coleman et al, 2008) Moreover due to the small

number of people involved in the project and the

organization, most of the management processes are

performed through an informal way and less

documented (O’Connor and Laporte, 2012). The

perception of heavyweight processes, especially in

terms of documentation, cost and nonalignment with

current development process, are among the reasons

why the companies did not plan to adopt a lifecycle

standard in the short to medium term (Basri et al,

2010) (Mora et al, 2011).

The definition of “Small” and “Very Small”

Entities is challengingly ambiguous, as there is no

commonly accepted definition of the terms. The

term “very small entity” (VSE) had been defined by

the ISO/IEC JTC1/SC7 Working Group 24 and

subsequently adopted for use in the new ISO/IEC

29110 process lifecycle standard as being “an entity

(enterprise, organization, department or project)

having up to 25 people” (Laporte et al, 2008).

Industry recognizes the value of Very Small

Entities (VSEs) in contributing valuable products

and services. A large majority of enterprises

worldwide are VSEs. A large majority of enterprises

Laporte C., O’Connor R. and García Paucar L..

Software Engineering Standards and Guides for Very Small Entities - Implementation in Two Start-ups.

DOI: 10.5220/0005368500050015

In Proceedings of the 10th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE-2015), pages 5-15

ISBN: 978-989-758-100-7

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

worldwide are VSEs. In Europe, for instance, as

illustrated in Table 1, over 92% of enterprises are

micro-enterprises. They have fewer than nine

employees. Micro enterprises account for 70% to

90% of enterprises in OECD countries and about

57% in USA. In Canada, close to 98 percent of

businesses are small businesses with fewer than 50

employees. About 32 percent of these have between

one and 19 employees (Statistics Canada 2008).

Table 1: Size of enterprises in Europe (Moll 2013).

Type Number of

Employees

Annual

turnover

No. of

enterprises

(% of overall)

Micro 1-9

≤2M

92.2

Small 10-49

≤10M

6.5

Medium 50-249

≤50M

1.1

Total

87 100 000 98.8

Large >250 >50M

Total

42 990 000 0.2

VSEs have unique characteristics, which make their

business styles different to larger organizations and

therefore most of the management processes are

performed through a more informal and less

documented manner (O’Connor et al, 2010).

Furthermore there is an acknowledged lack of

adoption of standards in small and very small

companies, as the perception is that they have been

developed for large software companies and not with

the small organisation in mind (O’Connor and

Coleman, 2009). As smaller software companies

have fewer resources in term of people and money

there are many challenges (Basri et al, 2011).

There is evidence that the majority of small and

very small software organizations are not adopting

existing standards / proven best practice models

because they perceive the standards as being

developed by large organizations and orientated

towards large organizations, thus provoking the

debate the in terms of number of employees, size

does actually matter (O'Connor and Coleman, 2006).

Studies have shown that small firms’ negative

perceptions of process model standards are primarily

driven by negative views of cost, documentation and

bureaucracy (Petkov et al, 2008). In addition, it has

been reported that SMEs find it difficult to relate

standards to their business needs and to justify the

application of the international standards in their

operations (O’Connor and Basri, 2012). Most SMEs

cannot afford the resources for, or see a net benefit

in, establishing software processes as defined by

current standards and maturity models (Coleman and

O’Connor, 2007).

Accordingly a new standard ISO/IEC 29110

“Lifecycle profiles for Very Small Entities” is aimed

at meeting the specific needs of VSEs (O’Connor

and Laporte, 2011a). The overall objective of this

new standard is to assist and encourage very small

software organizations in assessing and improving

their software process and it is predicted that this

new standard could encourage and assist small

software companies in assessing their software

development process. The approach (O’Connor and

Laporte, 2011b) used to develop ISO/IEC 29110

started with the pre-existing international standards,

such as the software life cycle standard

ISO/IEC/IEEE 12207 and the documentation

standard ISO/IEC/IEEE 15289.

There is a wide spectrum of development

approaches for organizations developing software.

Figure 1 illustrates the spectrum of approaches on 2

axes. The horizontal axis (from left to right)

illustrates the level of ceremony, from a low

ceremony approach with little documentation (e.g.

agile approach) to a high ceremony approach with a

comprehensive documentation (e.g. plan driven

CMMI

approach). The vertical axes illustrate the

approaches based on the level of risk. The top axis

illustrates a low risk linear approach using a

waterfall approach while the lower part of the axis

illustrates a risk-driven project using an iterative

approach. As we will explain below, ISO/IEC 29110

is located at about the centre of both axes.

Figure 1: Positioning of the ISO/IEC 29110 (adapted from

Kroll 2003).

The working group behind the development of this

standard is advocating the use of pilot projects as a

mean to accelerate the adoption and utilization of

ISO/IEC 29110 by VSEs (O’Connor and Laporte,

2010). Pilot projects are an important mean of

reducing risks and learning more about the

Low Ceremony

Waterfall

Few risks, sequential

Late integration and testing

High Ceremony

Iterative

Little documentation

Light process

XP, Scrum,

Adaptive

Development

Risk-driven

Continuons Integration and testing

Well-documented

Traceability

CCB

CMM

CMMI

29110

ENASE2015-10thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

organizational and technical issues associated with

the deployment of new software engineering

practices (Laporte et al, 2013a). To date a series of

pilot projects for the software engineering profile

standard have been completed in several countries

with the results published in a variety of literature

(Laporte et al, 2013b; O’Connor, 2012; Ribaud et al,

2010).

For most enterprises, but in particular for VSEs,

international certifications can enhance credibility,

competitiveness and access to national and

international markets. Brazil has developed an

ISO/IEC 29110 certification process. An ISO/IEC

29110 auditor should be competent in auditing

techniques, have expertise in ISO/IEC 29110 and

have experience in software development.

2 INTERNATIONAL STANDARDS

FOR VSES

2.1 Development

Since an international standard dedicated to the

software life cycle processes was already available,

i.e. ISO/IEC/IEEE 12207 (2008), WG24, the

ISO/IEC JTC1 SC7

working group mandated to

develop the new set of standards for VSEs, used the

concept of ISO standardized profiles (SP) to develop

the new standards for VSEs developing software.

From a practical point of view, a profile is a kind of

matrix, which identifies precisely the elements that

are taken from existing standards from those that are

not. The overall approach followed by WG24 to

develop this new standard for VSE consisted of the

following steps:

 develop a set of profiles for VSEs not

involved in critical software development,

 select the ISO/IEC/IEEE 12207 process

subsets applicable to VSEs having up to 25

people,

 select the description of the products, to be

produced by a project, using ISO/IEC/IEEE

15289 (2011) standard

 develop guidelines, checklists, templates,

examples to support the subsets selected.

2.2 Generic Profile Group

The basic requirements of a software development

process are that it should fit the needs of the project

and aid project success (Clarke 2011). And this need

should be informed by the situational context where

in the project must operate and therefore, the most

suitable software development process is contingent

on the context (Clarke et al 2012) (Jeners et al

2013). The core situational characteristic of the

entities targeted by ISO/IEC 29110 is size

Profile Groups are a collection of profiles. The

Generic Profile Group has been defined as

applicable to VSEs that do not develop critical

software. This Profile Group is a collection of four

profiles (Entry, Basic, Intermediate, Advanced)

providing a roadmap to satisfying a vast majority of

VSEs worldwide. VSEs targeted by the Entry Profile

are VSEs working on small projects (e.g. at most six

person-months effort) and for start-up VSEs. The

Basic Profile describes software development

practices of a single application by a single project

team of a VSE. The Intermediate Profile is targeted

at VSEs developing multiple projects with more than

one project team. The Advanced Profile is target to

VSEs which want to sustain and grow as a

competitive software development business.

The ISO/IEC 29110 standards and technical

reports targeted by audience. The set of documents

for the Basic profile (including ISO/IEC TR 29110-

5-1-2:2011 (2011) and ISO/IEC TR 29110-1:2011

(2011) were published in 2011. At the request of

WG24, all ISO/IEC 29110 TRs are available at no

cost from ISO. The Management and Engineering

Guide, the most valuable document for VSEs, has

being translated in French and in Spanish by Peru

and adopted as a Peruvian national standard. The set

of 5 documents has been translated in Portuguese by

Brazil and adopted as a Brazilian national standard.

The set of 5 documents has been translated in

Spanish by Uruguay and adopted as a national

standard. Japan has translated and adopted ISO/IEC

29110 as a Japanese national standard. The

Management and Engineering guide of the Entry

profile has been published in English, in French,

Portuguese and in Spanish.

2.3 Overview of the Basic Profile for

VSEs Developing Software

At the core of this standard is a Management and

Engineering Guide, officially know as ISO/IEC TR

29110-5-1-2, which focuses on Project Management

and Software Implementation as illustrated in Figure

1. The purpose of the Basic Profile is to define

Software Implementation (SI) and Project

Management (PM) processes from a subset of

ISO/IEC/IEEE 12207 and ISO/IEC/IEEE 15289

appropriate for VSEs, as illustrated in Figure 2.

The purpose of the Basic Profile is to define

SoftwareEngineeringStandardsandGuidesforVerySmallEntities-ImplementationinTwoStart-ups

Software Implementation (SI) and Project

Management (PM) processes from a subset of

ISO/IEC/IEEE 12207 and ISO/IEC/IEEE 15289

appropriate for VSEs. The main reason to include

project management is that the core business of

VSEs is software development and their financial

success depends on successful project completion

within schedule and on budget, as well as on making

a profit. The high-level view and the relationships

between the Software Implementation Process and

the Project Management processes are illustrated in

Figure 2.

Figure 2: Basic profile processes and activities (Laporte et

al, 2014c).

2.4 Structure of the Basic Profile

This standard defines two processes, Software

Implementation and Project Management. The

purpose of the Software Implementation process is

the systematic performance of the analysis, design,

construction, integration and tests activities for new

or modified software products according to the

specified requirements. The purpose of the Project

Management process is to establish and carry out in

a systematic way the tasks of the software

implementation project, which allows complying

with the project’s objectives in the expected quality,

time and cost.

The seven objectives of the PM process are:

 The Project Plan for the execution of the

project is developed according to the

Statement of Work and reviewed and accepted

by the Customer. The tasks and resources

necessary to complete the work are sized and

estimated.

 Progress of the project is monitored against

the Project Plan and recorded in the Progress

Status Record.

 The Change Requests are addressed through

their reception and analysis. Changes to

software requirements are evaluated for cost,

schedule and technical impact.

 Review meetings with the Work Team and the

Customer are held. Agreements are registered

and tracked.

 Risks are identified as they develop and

during the conduct of the project.

 A software Version Control Strategy is

developed. Items of Software Configuration

are identified, defined and baselined.

Modifications and releases of the items are

controlled and made available to the Customer

and Work Team including the storage,

handling and delivery of the items.

 Software Quality Assurance is performed to

provide assurance that work products and

processes comply with the Project Plan and

Requirements Specification.

The four activities of the Project Management

Process are:

 Project Planning: The primary objective of

this process is to produce and communicate

effective and workable project plans. This

process determines the scope of the project

management and technical activities, identifies

process outputs, project tasks and deliverables,

establishes schedules for project task conduct,

including achievement criteria, and required

re- sources to accomplish project tasks.

 Project Plan Execution: To implement the

actual work tasks of the project in accordance

with the project plan. Ideally when the project

plan has been agreed and communicated to all

teams members, work of the development of

the product, which is the subject of the

project, should commence.

 Project Assessment and Control: Purpose is to

determine the status of the project and ensure

that the project performs according to plans

and schedules, within projected budgets and it

satisfies technical objectives.

 Project Closure: Typically involves releasing

the final deliverables to the customer, handing

over project documentation to the business,

terminating supplier contracts, releasing

project resources and communicating project

closure to all stakeholders.

The purpose of the Software Implementation

process is to achieve systematic performance of the

analysis, design, construction, integration, and test

activities for new or modified software products

according to the specified requirements. The seven

objectives of the SI process are

 Tasks of the activities are performed through

ENASE2015-10thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

the accomplishment of the current Project

Plan.

 Software requirements are defined, analyzed

for correctness and testability, approved by the

Customer, baselined and communicated.

 Software architectural and detailed design is

developed and baselined. It describes the

Software Components and internal and

external interfaces of them.

 Software Components defined by the design

are produced. Unit test are defined and

performed to verify the consistency with

requirements and the design.

 Software is produced performing integration

of Software Components and verified using

Test Cases and Test Procedures. Results are

recorded at the Test Report.

 A Software Configuration, that meets the

Requirements Specification as agreed to with

the Customer, which includes user, operation

and maintenance documentations, is

integrated, baselined and stored at the Project

Repository.

 Verification and Validation Tasks of all

required work products are performed using

the defined criteria to achieve consistency

among output and input products in each

activity.

The activities of the Software Implementation

Process are:

 Software Implementation Initiation: Ensures

that the Project Plan established in Project

Planning activity is committed to by the Work

Team.

 Software Requirements Analysis: Analyzes

the agreed Customer’s requirements and

establishes the validated project requirements.

The activity provides:

 Software Architectural and Detailed Design:

Transforms the software requirements to the

system software architecture and software de-

tailed design.

 Software Construction: Develops the soft-

ware code and data from the Software Design.

 Software Integration and Tests: Ensures that

the integrated Software Components satisfy

the software requirements.

 Product Delivery: Provides the integrated

software product to the Customer.

As illustrated in figure 2, the customer’s

statement of work (SOW) is used to initiate the PM

process. The project plan will be used to guide the

execution of the software requirements analysis,

software architectural and detailed design, software

construction, and software integration and test, and

product delivery activities. The PM process closure

activity will deliver the Software Configuration (i.e.

a set of software products such as documentation,

code and tests) and will obtain the customer’s

acceptance to formalize the end of the project.

2.5 Development of Deployment

Packages

In order to facilitate the implementation, by VSEs,

of a Profile, a set of Deployment Packages (2013)

are available. A deployment package is a set of

artefacts developed to facilitate the implementation

of a set of practices, of the selected framework, in a

VSE. But, a deployment package is not a complete

process reference model. Deployment packages are

not intended to preclude or discourage the use of

additional guidelines that VSEs find useful.

The elements of a typical deployment package

are: technical description, relationships with

ISO/IEC 29110, key definitions, detailed description

of processes, activities, tasks, roles and products,

template, checklist, example, references and

mapping to standards and models, and a list of tools.

The mapping is only given as information to show

that a Deployment Package has explicit links to Part

5, ISO standards, such as ISO/IEC/IEEE 12207, or

models such as the CMMI developed by the

Software Engineering Institute. Hence by deploying

and implementing a package (O’Connor and

Sanders, 2013) a VSE can see its concrete step to

achieve or demonstrate coverage to Part 5.

A novel approach was taken to assist VSEs with

the deployment of ISO/IEC 29110 and to provide

guidance on the actual implementation this standard.

A set of Deployment Packages (DPs) have been

developed to define guidelines and explain in more

detail the processes defined in the ISO/IEC 29110

profiles (O’Connor and Laporte, 2014). The

elements of a typical DP are: description of

Figure 3: DPs support for Basic Profile (Laporte and

O’Connor, 2014b).

SoftwareEngineeringStandardsandGuidesforVerySmallEntities-ImplementationinTwoStart-ups

processes, activities, tasks, steps, roles, products,

templates, checklists, examples, references and

mapping to standards and models, and a list of tools.

DPs were designed such that a VSE can

implement its content, without having to implement

the complete ISO/IEC 29110 framework, i.e. all the

management and engineering activities, at the same

time. A set of nine DPs have been developed and are

freely available from (DP 2014). Figure 3 illustrates

the set of DPs developed to support the Basic

Profile.

3 IMPLEMENTATION TRIALS

The working group (ISO/IEC JTC1/SC7 WG 24)

behind the development of this standard is

advocating the use of pilot projects as a mean to

accelerate the adoption and utilization of ISO/IEC

29110. Pilot projects are an important means of

reducing risks and learning more about the

organizational and technical issues associated with

the deployment of new software engineering

practices.

In this section we will present 2 trial

implementations of ISO/IEC 29110 in IT start-ups.

The purpose of these trials is to illustrate the usage

of this standard in an industrial context and to

provide feedback to standards authors. Whilst not a

detailed methodological approach to validation of

this standard and whilst acknowledging the

validation limitations, we believe that these high

level results are useful to researchers and

practitioners alike.

3.1 Implementation in a Peruvian IT

Start-up

Over 98% of Perú are micro, small and medium

enterprises (MSMEs) having fewer than 10 workers.

About 7,6 million people work in companies having

fewer than 10 workers. About 14,000 Peruvian

companies are associated with the Information

Technology and Communications (ITC) industry

(Krasner, 1998).

An implementation of ISO/IEC 29110 has been

conducted in a four-people start-up VSE created in

2012 (Garcia et al, 2015). During its two years of

existence, the VSE has been involved in over 80

projects, most of which have lasted less than two

months. The VSE used agile practices to implement

software solutions such as Web 2.0 responsive

design systems and mobile applications. After

completing the implementation of the Basic profile

of ISO/IEC 29110, the VSE executed in 2014 a

project under contract. The product developed was a

software solution that facilitates communication

between clients and legal consultants at one of the

largest insurance companies in Peru. The solution

had to be implemented on a web platform and

deployed into a cloud environment.

Since the VSE was using agile methods to

implement its software projects, customer

requirements were expressed as user stories. For this

project, the VSE had determined that the duration of

a sprint would be one week. The project had 6

sprints. All software components, test cases, test

procedures and user stories were linked through a

traceability matrix. A subset of the traceability

matrix for a user story is shown in Figure 4.

Figure 4: Subset of traceability matrix for one user story

(Garcia et al 2015).

As illustrated in Table 2, the total effort to

implement the project was 882 hours. The effort

devoted to prevention activities such as installation

of the environment (servers, tools, etc.) was 14

hours, task execution took 585 hours, reviews took

124 hours and effort to correct defects identified in

reviews and in testing took 159 hours. The start-up

wasted only 18% of the total project effort (i.e. 159

hours/882 hours) on rework.

Since it was the first time the VSE had executed

the new ISO/IEC 29110 processes in a real project,

so there was a learning curve that resulted in

additional hours spent on rework for different

project tasks. Despite this situation, the result was

ENASE2015-10thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

close to the percentage of rework (i.e. about 15% to

25%) of an organization that has implemented the

Capability Maturity Model and is at maturity level 3.

Table 2: effort to execute, detect and correct errors (Garcia

et al 2015).

Title of task

Prevention

(hours)

Execution

(Hours)

Review

(Hours)

Rework

(Hours)

Environment

installation

Project plan

development

15 3 7

Project plan

execution

and project

assessment

& control

108

Specification

development

107 28 58

Architecture

development

35 10 14

Test plan

development

45 8 11

Code

development

and testing

253 70 62

Develop

user guide &

maintenance

document

14 5 7

Product

deployment

Project

closure

Total hours 14 585 124 159

As illustrated in Figure 5, the ISO/IEC 29110

certification process is composed of four stages. In

the first stage, a VSE applies for an ISO/IEC 29110

audit and if it is successful, a commercial and

technical agreement is entered into with the

accreditation body. Then, the initial certification

audit begins. If the audit is successful, a three-year

initial certificate is issued by a national accreditation

body. In this case, the certificate was issued by the

Brazilian national accreditation body.

For the first stage of the audit process, the

Peruvian VSE invested about 22 hours. For the

initial certification stage, the VSE invested about 63

hours. The cost of the auditor, excluding the travel

expenses, was 1,500$. The total effort and cost of an

ISO/IEC 29110 audit is very small compared to a

typical CMMI official assessment. This start-up

became the first Peruvian VSE to obtain an ISO/IEC

29110 certification.

The third stage of a certification cycle involves

the completion of two surveillance audits one and

two years after obtaining the initial certification.

Finally, the fourth stage is the recertification of the

VSE; once the 3-year certification cycle has elapsed.

Figure 5: ISO/IEC 29110 certification process (Laporte et

al 2014d).

In order to promote the recognition of qualifications

between countries, there are international

organizations such as the International Accreditation

Forum (IAF). The IAF is the world association of

conformity assessment accreditation bodies in the

fields of management systems, products and

services, and to date, it has more than 60 member

countries. The Peruvian and the Brazilian

accreditation bodies are members of this

organization. An ISO/IEC 29110 certificate of

conformity issued by an accreditation body member

of the IAF is recognized by all members of IAF. The

conformity certificate has become a major

differentiator with regard to the main competitors of

the VSE. The Peruvian start-up VSE has gained

access to larger software development projects and

increased its customer base. The VSE has increased

its number of workers to date, from 4 to 10

employees.

3.2 Implementation in a Canadian IT

Start-up

An implementation project has been conducted in an

IT start-up VSE by a team of two (part-time)

developers (Laporte et al, 2014c). Their web

application allows users to collaborate, share and

plan their trips simply and accessible to all. The use

of the Basic profile of ISO/IEC 29110 has guided

the start-up to develop an application of high quality

while using proven practices of ISO 29110. The total

effort of this project was nearly 1000 hours. The two

members of the team were assigned roles and

activities of ISO 29110.

During the software development, a traceability

matrix was developed between the software

requirements, defined in the requirements

specification document, and the software

SoftwareEngineeringStandardsandGuidesforVerySmallEntities-ImplementationinTwoStart-ups

components. Since, in most projects requirements,

defined in the requirements activity, are never

finalized at the end of this activity, a traceability

matrix is very useful. One advantage of such a

matrix is the possibility of rapidly identifying the

impacted software components when modifications,

additions, deletions, of software requirements are

done during a project.

Verification tasks, such as peer reviews, were

performed on documents such as the requirement

specifications and the architecture. The team used

the desk-check to review their documents which is

inexpensive and easy to implement in any

organization and can be used to detect anomalies,

omissions, improve a document or present and

discuss alternative solutions.

As defined in ISO/IEC 29110, the software

integration and tests activity ensures that the

integrated Software Components satisfy the software

requirements. This activity provides (ISO 2011c):

Work team review of the project plan to

determine task assignment.

 Understanding of test cases and procedures

and the integration environment.

 Integrated software components, corrected

defects and documented results.

 Traceability of requirements and design to the

integrated software product.

 Documented and verified operational and

software user documentations.

 Verified software baseline.

To manage the defects detected, a tracking tool

was used. Such software allowed the team to do an

inventory of problems found during the integration

and testing activity, to track problems and to classify

them, and to determine a priority for each defect

found. In this project, the open source Bugzilla

software tool had been used to manage the defects.

The test report presents the results of tests carried

out using the test plan. These results are used to

illustrate the number of problems found and the

progress of the resolution of anomalies. The test plan

Table 3: Number and types of defects detected through

testing and corrected (Laporte et al 2014c).

Seriousness No. of

defects

detected

No. of

defects

corrected

% of

defects

corrected

Blocker 3 3 100%

Critical 22 22 100%

Major 11 11 100%

Normal 12 12 100%

Minor 19 6 32%

includes 112 cases which have been successfully

completed with the exception test cases connected to

one type of defect: the validation of the date format

when manually entered by a user. Since this defect

was classified as "minor", it was decided not to

correct their instances during the first cycle of

development. Table 3 illustrates the percentage of

defects detected during the execution of the tests for

each category of defects.

The defects classified by severity using the

following defect classification:

 Blocker: prevents function from being used,

no work-around, blocking progress on

multiple fronts

 Critical: prevents function from being used, no

work-around

 Major: prevents function from being used, but

a work-around is possible

 Normal: a problem making a function difficult

to use but no special work-around is required

 Minor: a problem not affecting the actual

function, but the behaviour is not natural

The members of the start-up have recorded the

effort, in person-hours, spent on tasks of the project

to the nearest 30 minutes. Table 4 shows, for each

major task, the effort to execute the task, the effort

required to review a document, such as the software

specification document, in order to detect errors and,

the effort required to correct the errors (i.e. the

rework). As an example, for the development of the

software architecture document, it took 42.5 hours to

develop, an additional 1.5 hour to conduct a review

and an additional 3.5 hours to correct the errors.

As illustrated in table 4 for this start-up project,

about 8.9% (i.e. 89 hours/990.5 hours) of the total

project effort has been spent in prevention tasks such

as the installation of the server, the workstations and

the software tools; and only 12.6% has been spent

on rework (i.e. 125 hours/990.5 hours). This

indicates that the use of appropriate standards, in this

case for a start-up company, can guide all the phases

of the development of a product such that the wasted

effort (i.e. rework) is about the same as a more

mature organization (i.e. about level 3 of CMM).

In most start-ups, the wasted effort, for a project

similar to this one, would have added about 90 hours

(i.e. 30% of 716 or 215 hours – 125 hours). This also

implies, that for a net effort of about 6 hours per

member per day (if we subtract from an 8-hour day

interruptions (e.g. phone call), answering emails,

discussions in corridors, etc.), the product would

have been ready for delivery to a customer about 15

days, of 6 hours, later than with a project with only

12.6% of waste.

ENASE2015-10thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

Table 4: Effort to execute, detect and correct errors by the

2-member team (Laporte et al 2014c).

Title of

task

Prevention

(hours)

Execution

(Hours)

Review

(Hours)

Rework

(Hours)

Environmen

t installation

Project plan

developmen

35 3 4

Project plan

execution

and project

assessment

& control

Specificatio

n &

prototype

developmen

199.5 7 18

Architecture

developmen

42.5 1.5 3.5

Test plan

developmen

12.5 1 2

Code

developmen

t and testing

361 47 96.5

Develop

user guide

maintenance

document

8 1 1

Web site

deployment

8.5

Project

closure

Total hours 89 716 60.5 125

These two projects have demonstrated that, by using

ISO/IEC 29110, it was possible to properly plan the

project and develop the software product using

proven software practices documented in standards

as well as not interfering with the creativity during

the development of their web site. People who think

that standards are a burden, an unnecessary overhead

and a treat to creativity should look at this start-up

project and revisit their results.

4 CONCLUSIONS AND FUTURE

WORK

The relationship between the success of a software

company and the software process it utilized has

been investigated (Laporte and O’Connor, 2014a)

(O’Connor and Basri, 2014) showing the need for all

organizations, not just VSEs to pay attention to

software process practices such as ISO standards. As

ISO/IEC 29110 is an emerging standard there is

much work yet to be completed. The main remaining

work item is to finalize the development of the

remaining two software profiles of the Generic

Profile Group: (a) Intermediate - management of

more than one project and (b) Advanced - business

management and portfolio management practices.

The ISO working group, initially mandated to

develop the ISO/IEC 29110 for software, was also

mandated to develop a similar approach for VSEs

involved in systems engineering (Laporte and

O’Connor, 2014b). In August 2014, ISO published

the ISO/IEC 29110 systems engineering and

management guide of the Basic profile ISO/IEC TR

29110-5-6-2:2014 (2014). The systems engineering

and management guide of the Entry profile has been

published in 2015 as ISO/IEC TR 29110-5-6-1:2015

(2015).

With any new initiative there is much to be

learned from conducting pilot projects. One issue of

major importance to VSEs which is emerging from

these pilot projects and similar work by the ISO

working group is the need for a light-weight flexible

approach to process assessment. Whilst work is

currently underway on an assessment mechanism for

ISO/IEC 29110 (ISO, 2011b), a clear niche market

need is emerging which may force the process

assessment community to change their views on how

process assessments are carried out for VSEs. In

particular there is a strong need to ensure that VSEs

are not required to invest anything similar in terms

of time, money and other resources on process

assessments, as may be expected from their larger

SMEs (small and medium enterprises), or even

MNC (multinational corporations) counterparts.

Indeed some form of self-assessment, possibly

supported by Internet based tools, along with

periodic spot-checks may be a suitable alternative to

meet the unique needs of VSEs. It is clear that the

process assessment community will have to rethink

process assessment, new methods and ideas for

assessing processes in VSEs.

It is expected that some VSEs will use the

technology developed on their own, other VSEs will

get some help from government organizations, such

as training or coaching, and some large

organizations will impose the ISO/IEC 29110

standards on the VSEs that supply components for

their products. A few countries have opted for the

‘survival of the fittest’ strategy for their VSEs, i.e.

SoftwareEngineeringStandardsandGuidesforVerySmallEntities-ImplementationinTwoStart-ups

an approach where a government does not intervene

in the marketplace and lets the market decide which

VSEs will survive. At the same time, a number of

government agencies, universities, research centers

and associations are working to determine how to

help VSEs.

5 ADDITIONAL INFORMATION

The following web site provides more information:

http://profs.logti.etsmtl.ca/claporte/English/VSE/ind

ex.html

REFERENCES

Basri, S., & O'Connor, R. V. 2010. Evaluation on

knowledge management process in very small

software companies: a survey, 5th Knowledge

Management International Conference, Terengganu,

Malaysia, May 2010.

Basri S, O’Connor RV. 2011 A study of software

development team dynamics in SPI. Systems, Software

and Services Process Improvement (EuroSPI 2011),

CCIS, vol. 172, pp. 143-154. Springer-Verlag.

Basri, S., & O'Connor, R. (2012). A study of knowledge

management process practices in very small software

companies. American Journal of Economics and

Business Administration, 3(4), 636-644.

Coleman, G., O’Connor, R. 2006. Software Process in

Practice: A Grounded Theory of the Irish Software

Industry. In: Richardson, I., Runeson, P., Messnarz, R.

(eds.) EuroSPI 2006. LNCS, vol. 4257, pp. 28–39.

Springer, Heidelberg.

Coleman, G., & O'Connor, R. V. (2008). An investigation

into software development process formation in

software start-ups. Journal of Enterprise Information

Management, 21(6), 633-648.

Clarke, P., & O’Connor, R. 2010. Harnessing ISO/IEC

12207 to Examine the Extent of SPI Activity in an

Organisation. In Systems, Software and Services

Process Improvement (pp. 25-36). Springer Berlin

Heidelberg.

Clarke, P., O'Connor, R.V. 2011. The Meaning of Success

for Software SMEs: An Holistic Scorecard Based

Approach. Systems, Software and Services Process

Improvement (EuroSPI 2011), CCIS, vol. 172, pp.

272-83. Springer-Verlag, Heidelberg.

Clarke, P. and O'Connor, R., The situational factors that

affect the software development process: Towards a

comprehensive reference framework, Journal of

Information and Software Technology, Vol. 54, Issue

5, May 2012. pp. 433-447.

DP 2014. Deployment Packages repository, (online)

available from: http://profs.logti.etsmtl.ca/claporte/

English/VSE/index.html.

Krasner, H. 1998. Using the cost of quality approach for

software. Crosstalk – The Journal of Defense Software

Engineering 11 (November):6-11.

Garcia, L., Laporte, C.Y., Arteaga, J., Bruggmann, M.,

Implementation and Certification of ISO/IEC 29110 in

an IT Startup in Peru, Software Quality Professional

Journal, ASQ, vol. 17, no. 2, 2015.

ISO/IEC/IEEE 12207:2008, Systems and software

engineering– Software life cycle processes.

International Organization for

Standardization/International Electrotechnical

Commission: Geneva, Switzerland.

ISO/IEC/IEEE 15289:2011, Systems and software

engineering - Content of systems and software life

cycle process information products (Documentation),

International Organization for

Standardization/International Electrotechnical

Commission: Geneva, Switzerland.

ISO/IEC 29110-4-1:2011, Software Engineering --

Lifecycle Profiles for Very Small Entities (VSEs) -

Part 4-1: Profile specifications: Generic profile group.

Geneva: International Organization for

Standardization (ISO), 2011.

ISO/IEC TR 29110-1:2011, “Software Engineering -

Lifecycle Profiles for Very Small Entities (VSEs) -

Part 1: Overview”. Geneva: International

Organization for Standardization (ISO), 2011.

Available at no cost from ISO at:

http://standards.iso.org/ittf/PubliclyAvailableStandard

s/c051150_ISO_IEC_TR_29110-1_2011.zip.

ISO/IEC TR 29110-5-6-2:2014 - Systems Engineering –

Lifecycle Profiles for Very Small Entities (VSEs) -

Management and engineering guide: Generic profile

group: Basic profile, International Organization for

Standardization/International Electrotechnical

Commission: Geneva, Switzerland. Available at no

cost from ISO at: http://standards.iso.org/ittf/

PubliclyAvailableStandards/c063371_ISO_IEC_2911

0-5-6_2_2014.zip.

ISO/IEC TR 29110-5-6-1:2015 - Systems Engineering –

Lifecycle Profiles for Very Small Entities (VSEs) -

Management and engineering guide: Generic profile

group: Entry profile, International Organization for

Standardization/International Electrotechnical

Commission: Geneva, Switzerland. Available at no

cost from ISO at: http://standards.iso.org/ittf/

PubliclyAvailableStandards/index.html.

Jeners, S., Clarke, P., O’Connor, R. V., Buglione, L., and

Lepmets, M. Harmonizing Software Development

Processes with Software Development Settings – A

Systematic Approach, In McCafery, F., O'Connor,

R.V. and Messnarz R. (Eds), Systems, Software and

Services Process Improvement, CCIS 364, Springer-

Verlag, 2013.

Laporte, C. Y., O’Connor, R. V. 2014a. A Systems Process

Lifecycle Standard for Very Small Entities:

Development and Pilot Trials, In Barafort, B.,

O'Connor, R.V. and Messnarz R. (Eds), Systems,

Software and Services Process Improvement, CCIS

Vol. 425, Springer-Verlag.

ENASE2015-10thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

Laporte, C.Y.; O'Connor, Rory V. 2014b. Systems and

Software Engineering Standards for Very Small

Entities: Implementation and Initial Results. 9th

International Conference on the Quality of

Information and Communications Technology

(QUATIC), pp.38-47, 23-26 Sept.

Laporte, C.Y., Hébert, C., Mineau, C., 2014c.

Development of a Social Network Website Using the

New ISO/IEC 29110 Standard Developed Specifically

for Very Small Entities, Software Quality Professional

Journal, ASQ, vol. 16, no. 4, pp 4-25, 2014.

Laporte, C.Y., Houde, R., Marvin, J., 2014d. Systems

Engineering International Standards and Support

Tools for Very Small Enterprises, Paper to be

presented at the 24th Annual International Symposium

of INCOSE (International Council on Systems

Engineering), Las Vegas, US, June 30th-July 3, 2014.

Laporte, C.Y., Séguin, N., Villas Boas, G., 2013b. Seizing

the benefits of software and systems engineering

standards, ISO Focus+, International Organization for

Standardization, February 2013, pp 32-36.

Laporte, C.Y., O'Connor, R., Fanmuy, G., 2013a.

International Systems and Software Engineering

Standards for Very Small Entities, CrossTalk - The

Journal of Defense Software Engineering, May/June

2013, Vol. 26, No 3, pp 28-33.

Laporte, C.Y., Alexandre, S., and O'Connor, R. 2008. A

Software Engineering Lifecycle Standard for Very

Small Enterprises, R.V. O'Connor et al (Eds)

Proceedings of EuroSPI Springer-Verlag, CCIS Vol.

16, pp. 129-141,

Mora, M., O'Connor, R., Raisinghani, M., & Macías-

Luévano, J. 2011. An IT service engineering and

management framework (ITS-EMF). International

Journal of Service Science, Management, Engineering,

and Technology, 2(2), 1-15.

Moll, R., 20134. Being prepared – A bird’s eye view of

SMEs and risk management, ISO Focus+, Geneva,

Switzerland: International Organization for

Standardization, February 2013.

O’Connor, R. V., & Coleman, G. 2007. An investigation

of barriers to the adoption of software process best

practice models. ACIS 2007 Proceedings, 35.

O'Connor R. and Coleman G., 2009. Ignoring ‘Best

Practice': Why Irish Software SMEs are rejecting

CMMI and ISO 9000, Australasian Journal of

Information Systems, Vol. 16, No. 1,

O'Connor, R., Basri, S. and Coleman, G., 2010. Exploring

Managerial Commitment towards SPI in Small and

Very Small Enterprises, in Riel et al (Eds), Systems,

Software and Services Process Improvement, CCIS

Vol. 99, Springer-Verlag, pp. 268-278.

O'Connor R. and Laporte, C.Y., 2010. Towards the

provision of assistance for very small entities in

deploying software lifecycle standards. In Proceedings

of the 11th International Conference on Product

Focused Software (PROFES '10). ACM.

O'Connor, R. and Laporte, C.Y., 2011a. Deploying

Lifecycle profiles for Very Small Entities: An Early

Stage Industry View, Proceedings of 11th

International SPICE Conference on Process

Improvement and Capability dEtermination, CCIS

Vol. 155, Springer-Verlag, May 2011.

O'Connor, R. and Laporte, C.Y., 2011b. Using ISO/IEC

29110 to Harness Process Improvement in Very Small

Entities, Workshop on SPI in SMEs,

18th European

Software Process Improvement Conference, CCIS Vol.

172, Springer-Verlag.

O’Connor, R. 2012. Evaluating Management Sentiment

Towards ISO/IEC 29110 in Very Small Software

Development Companies. In: Mas, et al. (eds.)

Software Process Improvement and Capability

Determination. CCIS, vol. 290, pp. 277–281. Springer-

Verlag, Heidelberg.

O’Connor, R. V., & Laporte, C. Y., 2012. Software project

management in very small entities with ISO/IEC

29110 (pp. 330-341). Springer Berlin Heidelberg.

O'Connor, R., & Basri, S., 2012. The effect of team

dynamics on software development process

improvement. International Journal of Human Capital

and Information Technology Professionals, 3(3), 13-

26.

O’Connor, R. V., & Sanders, M. 2013. Lessons from a

pilot implementation of ISO/IEC 29110 in a group of

very small Irish companies. In Software Process

Improvement and Capability Determination (pp. 243-

246). Springer Berlin Heidelberg.

O'Connor, R. V., and Basri, S. 2014. Understanding the

role of knowledge management in software

development: a case study in very small companies,

International Journal of Systems and Service-Oriented

Engineering, Vol. 4, No. 1, pp. 39-52.

O'Connor, R. V. and Laporte, C. Y., 2014. An Innovative

Approach to the Development of an International

Software Process Lifecycle Standard for Very Small

Entities, International Journal of Information

Technology and the Systems Approach, Vol. 7, No. 1,

pp. 1-22.

Petkov, D., Edgar-Nevill, D., Madachy, R., & O’Connor,

R. 2008. Information systems, software engineering,

and systems thinking: Challenges and opportunities.

International Journal of Information Technologies and

Systems Approach (IJITSA), 1(1), 62-78.

Ribaud, V., Saliou, P., O’Connor, R., Laporte, C.Y.: 2010.

Software Engineering Support Activities for Very

Small Entities. In: Riel, et al. (eds.) Systems, Software

and Services Process Improvement. CCIS, vol. 99, pp.

165–176. Springer-Verlag, Heidelberg.

Statistics Canada. 2008. Available at: http://www.ic.gc.ca/

sbstatistics.

SoftwareEngineeringStandardsandGuidesforVerySmallEntities-ImplementationinTwoStart-ups