PROFILING THE EFFORT OF NOVICES IN SOFTWARE

DEVELOPMENT TEAMS

An Analysis using Data Collected Non Invasively

Ilenia Fronza and Jelena Vlasenko

Free University of Bolzano – Bozen, Piazza Domenicani – Domenikanerplatz, 3, I-39100 Bolzano – Bozen, Italy

Keywords: Novices integration, Tool usage, Browsing purposes, AISEMA systems.

Abstract: New developers exhibit working patterns that are different from existing and experienced software

developers. Understanding such patterns may help in determining the actual level of introduction developers

have within a company. Moreover, browsing the internet has achieved a pivotal role, as browsing people

collect valuable information about working issues and, indeed, developers spend a significant amount of

effort browsing. Models have been proposed and validated for the introduction of novices in companies.

This paper analyses one of the most promising of these models, the one proposed by (Fronza et al., 2009),

and validates it against the patterns of usage of browsing. It appears that the patters of use of the browser

confirm the proposed model and that at the end of the observation period, when according to the model, the

new developers should have been fully introduced into the working patterns of the company, there is a

substantial congruence between the working patterns of the new developers and of the existing developers.

1 INTRODUCTION

It is widely known (Brooks, 1995) that novices may

slow down progress if not bring it to a complete stop

while they are brought up to speed. Despite of that,

important opportunities may be missed in software

industry if the team cannot increase the rate at which

novices complete functionality (Ronchetti et al.,

2006; Wei and Tai, 2010). In order to become

productive novices need to learn, just to give some

examples, programming languages, tools,

development processes, team dynamics, and coding

standards. The analysis of novices activities can

propose early insights on their integration in the

team. To date, there is a lack of evidences coming

from industrial teams, and analysed data are mostly

collected using time consuming and error prone

techniques (Sillitti et al., 2005).

In this work we address these issues by analysing

novices integration in an industrial team of

professional software developers. We use data

collected by an AISEMA system (PROM) (Coman

and Sillitti, 2007; Coman et al., 2008; Coman et al.,

2009; Scotto et al., 2006; Sillitti et al., 2003; Sillitti

et al., 2004) to:

• Compare novices and experts tool usage;

• Find the purpose of novices browsing activity.

We noticed that the developers, both experts and

novices, devoted a significant part of their effort to

the 3 following tools: Visual Studio, Outlook, and

Browser. There are no doubts about the purpose of

using Visual Studio. Moreover, we know that the

developers get their requirements and tasks via e-

mail what explains the effort devoted to Outlook.

Still, it is unknown what the developers do when

they browse. We assume that the developers use

Browser because of two reasons: browsing for

business purposes and browsing for private

purposes. We extracted keywords from headers of

visited web pages to assess whether the developers

were browsing for business or private purposes.

The analysis is performed for each of the four

integration phases proposed in (Fronza et al., 2009),

where developers interactions caused by Pair

Programming (PP) sessions have been analysed and

a model has been proposed for describing integration

of novices into a team. These four phases are: 1)

initiation, 2) independence, 3) maturity, and 4)

integration.

The paper is structured as follows: in Section 2,

we present some related work; in Section 3, we

introduce our approach. In Section 4, results are

presented. In Section 5 we discuss results and

limitations of this work. In Section 6, we draw

conclusions and discuss future work.

392

Fronza I. and Vlasenko J..

PROFILING THE EFFORT OF NOVICES IN SOFTWARE DEVELOPMENT TEAMS - An Analysis using Data Collected Non Invasively.

DOI: 10.5220/0003505203920395

In Proceedings of the 13th International Conference on Enterprise Information Systems (ICEIS-2011), pages 392-395

ISBN: 978-989-8425-55-3

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

2 RELATED WORK

Novices Work. In the first period of work, novices

have to learn about the problem domain and

software system, and also to adapt to the new

working environment (Sim and Holt, 1998). During

their first period of work in a team, novices have

been reported to be less efficient in looking for

information and in understanding it (Zou and

Godfrey, 2008), and less accurate and slower in

performing low-level programming tasks. According

to results in (Berlin, 1993), novices do more errors

and are easily misled by unclear error messages, use

a limited set of tools, wait longer than experts before

asking for help, and make a more complex and

error-prone plan for simple tasks.

Analysis of Tool Usage. Available works on tool

usage mainly focus on examining the use of tools to

improve them. (Lethbridge and Singer, 1997)

analyses tools used by developers to find how to

increase developers productivity. Both the studies

find developers to spend most of their time

navigating through source code. Therefore, the

proposal is to develop new tools to ease the

navigation process. (Coman et al., 2009) is one of

the most recent studies. Tool usage and purposes of

use are analysed in a team of three members. Each

developer is found to use from 11 to 13 tools, with

five purposes of usage: Documents, Navigating,

Communication, Internet, and Coding.

3 STRUCTURE OF THE STUDY

3.1 Data Collection and Environment

Data for this study were collected non-invasively by

means of PROM (PRO Metrics) (Sillitti et al., 2003)

from a team of professional software developers

working in an IT department of a large Italian

manufacturer that prefers to remain anonymous. The

length of the study is approximately 10 months:

from October 2007 to July 2008.

The team was composed of 17 developers: 15

existing team members working for the company

more than 5 years and 2 new team members who

joined the team before we started collecting the data.

The developers all hold university degrees in

computer-related areas and have programming

experience from 10 to 15 years.

3.2 Data Analysis

In the first part of this study we identified all the

tools that were used by all the developers during the

period of study. We found that the developers were

using 26 tools. Only 9 tools were used regularly and

by all the developers during the observation period:

Browser, Messenger, Excel, Word, Windows

Explorer, Management Console, Outlook, Remote

Desktop, and Visual Studio. The effort devoted to

these tools was more than 80% of the total effort.

We introduced the following formula to compare

novices and experts tool usage:

UC =

− U

max(U

)

(1)

Where U

is novices tool usage, U

is experts tool

usage, and UC (Usage Comparison) is the

comparison between these two variables. We use

radar plot to visualize how differently experts and

novices distribute their effort among tools.

We found that the developers devoted a

noticeable part of their time to browse Internet.

Thus, we decided to investigate the purposes of

using Browser. We assume that Browser can be used

because of 2 reasons:

• Business Browsing – when the developers go to

Internet to search for necessary information

they need to complete their tasks

• Private Browsing – when the developers go to

Internet to do their private business, i.e.,

internet shopping, reading news, youtube, etc.

Figure 1: Distribution of effort among 9 tools.

We used headers of the visited web pages to

identify a set of keywords that correspond to

PROFILING THE EFFORT OF NOVICES IN SOFTWARE DEVELOPMENT TEAMS - An Analysis using Data

Collected Non Invasively

393

Business Browsing. Based on these keywords we

extracted the effort devoted to Business Browsing

and Private Browsing.

4 RESULTS

Figure 1 represents distribution of effort among the

9 selected tools during a period of 10 months. We

notice that the developers, both experts and novices,

devote a significant part of their effort to the 3

following tools: Visual Studio, Outlook, and

Browser.

To understand better how differently experts and

novices use tools we compute UC for the four

phases. Figure 2 visualizes these results.

Figure 2: Comparison of experts and novices tool usage in

the four phases identified in (Fronza et al., 2009).

Based on the analysis of Figure 2 we make

the following assumptions:

• In the first phase, UC is higher than 0.5 in five

out of nine tools. Thus, we can assume that tool

usage is significantly different between experts

and novices;

• In the second phase, UC is lower than in the

first phase in eight out of nine tools. Four tools

(Visual Studio, Remote Desktop, Microsoft

Management Console, and Microsoft

Messenger) still have an associated UC around

0.4;

• Starting from the third phase, novices begin to

become more similar to experts in terms of tool

usage. In particular, there is no difference in

usage of Visual Studio and Outlook;

• In the final fourth phase, there is almost no

difference in usage of Visual Studio, Outlook,

and Microsoft Management Console. UC is

still higher than 0.4 only in two tools: Remote

Desktop and Microsoft Office Word;

• Browser has associated UC higher than 0.6 in

the first phase; in the 2nd, 3rd, and 4th phases

UC is stable and is between 0.2 and 0.4.

Considering the results from Figure 1 and Figure

2 we notice that the developers spend a

noticeable part of their time browsing Internet

and there is a difference in how experts and

novices use Browser.

Figure 3 indicates that both experts and novices

when they browse devote most of their time

browsing for business purposes. Though, it can be

noticed that in the 1

and 2

phases experts devote

significantly more time to Business Browsing than

novices. The results may indicate that when

developers go to Internet during their working time

the reason in most of the cases is a lack of necessary

knowledge.

Figure 3: Percentage of novices and experts business

browsing in the four phases identified in (Fronza et al.,

2009).

5 CONCLUSIONS

In this study we analysed how experts and novices

use tools during a period of 10 months.

We found that most time consuming applications

are Visual Studio, Outlook, and Browser. We

confirmed in our study the four integration phases

introduced in (Fronza et al., 2009). We found that in

the 1

phase experts and novices use tools in a very

different way and then as time passes the difference

also decreases. Moreover, we noticed that both

experts and novices spend a part of their time

browsing Internet.

We divided browsing into two categories:

Business Browsing and Private Browsing and based

on keywords extracted data to investigate how much

time the developers devote to each category of

browsing. We found that most of the browsing time

the developers visit web pages connected to their

work in order to get necessary knowledge. We

assume that in this case Browser should not be

counted as a distracting activity but as a part of

working process.

Furthermore, we studied how usage of Browser

changes throughout the four phases and noticed that

in the first two phases the developers browse more

ICEIS 2011 - 13th International Conference on Enterprise Information Systems

394

for private purposes and then they start using

Browser more business purposes, meaning that they

start behaving similar to experts and are integrated

to the team.

Overall, we think that it is an interesting topic

to investigate since browsing is an activity that

consumes a lot of time and its benefit is often

questioned by managers.

REFERENCES

Berlin, L. M. 1993. Beyond program understanding: a

look at programming expertise in industry. In 5th

Workshop on Empirical Studies of Programmers, pp.6

-- 25.

Brooks, F. P. Jr. 1995. The Mythical Man Month,

Addison-Wesley.

Coman I. And Sillitti A. 2007. An Empirical Exploratory

Study on Inferring Developers' Activities from Low-

Level Data. In SEKE’07, International Conference on

Software Engineering and Knowledge Engineering.

Coman, I. D., Sillitti, A., and Succi, G. 2008. Investigating

the Usefulness of Pair-Programming in a Mature Agile

Team. In Proceedings of the International Conference

on Agile Porcesses and eXtreme Programming in

Software Engineering (Limerick, Ireland, June 10-14,

2008), 127-136.

Coman, I. D., Sillitti, A., and Succi, G. 2009. A case-study

on using an Automated In-process Software

Engineering Measurement and Analysis system in an

industrial environment. In ICSE’09, International

Conference on Software Engineering, pp. 89 – 99.

Fronza, I., Sillitti, A., Succi, G. 2009. An interpretation of

the results of the analysis of pair programming during

novices integration in a team. In ESEM’09, 3rd

International Symposium on Empirical Software

Engineering and Measurement, pp. 225 -- 235.

Lethbridge, T. C. and Singer, J. 1997. Understanding

Software Maintenance Tools: Some Empirical

Research. In Workshop on Empirical Studies of

Software Maintenance, pp. 157 – 162.

Moser R., Sillitti A., Abrahamsson P., and Succi G. 2006.

Does refactoring improve reusability? In Proceedings

of the International Conference on Software Reuse ,

pp. 287-297, 2006.

Ronchetti, M., Succi, G., Pedrycz, W., and Russo, B.

2006. Early Estimation of Software Size in Object-

Oriented Environments. A Case Study in a CMM level

3 Software Firm. Information Sciences, 176, 2006,

475-489.

Scotto M., Sillitti A., Succi G., Vernazza T. 2004. A

Relational Approach to Software Metrics. In

Proceedings of the Symposium on Applied

Computing, pp. 1536-1540, 2004.

Scotto M., Sillitti A., Succi G., Vernazza T. 2006. A Non-

Invasive Approach to Product Metrics Collection.

Journal of Systems Architecture, 52, 11, pp. 668 –

675.

Sillitti, A., Janes, A., Succi, G., and Vernazza, T. 2003.

Collecting, Integrating and Analyzing Software

Metrics and Personal Software Process Data. In

EUROMICRO, pp. 336 – 342.

Sillitti A., Janes A., Succi G., Vernazza T. 2004. Measures

for Mobile Users: an Architecture. Journal of Systems

Architecture, 50, 7, pp. 393 – 405.

Sillitti A., Ceschi M., Russo B., Succi G. 2005. Managing

Uncertainty in Requirements: a Survey in Plan-Based

and Agile Companies. In Proceedings of the

International Software Metrics Symposium, 19-22,

2005.

Wei, G. and Tai, L. 2010. An Empirical Study on

Organizational Commitment and Turnover of IT

Industry. In International Conference on E-Business

and E-Government, pp. 904 -- 906.

Zou, L. and Godfrey, M. W. 2008. Understanding

Interactions Differences Between Newcomer and

Expert Programmers. In International Workshop on

Recommendation Systems for Software, pp. 26 – 29.

PROFILING THE EFFORT OF NOVICES IN SOFTWARE DEVELOPMENT TEAMS - An Analysis using Data

Collected Non Invasively

395