WEB METRICS SELECTION THROUGH A PRACTITIONERS’
SURVEY
Julian Ruiz, Coral Calero, Mario Piattini
Alarcos Research Group, Information Systems and Technologies Department, UCLM-Soluziona Research and
Development Institute. University of Castilla-La Mancha, Paseo de la Universidad, 4, 13071 - Ciudad Real, Spain
Keywords: Web Metrics, Quality.
Abstract: There are a lot of web metrics proposals. However, most previous work does
not include their practical
application. The risk of doing so, is to limit all the effort made just to an academic exercise. In order to
eliminate this gap as well as to be able to apply the work developed, it is necessary to involve the different
stakeholders related to web technologies as an essential part of web metrics definition. So, it is crucial to
know the perception they have about web metrics, especially those related to the development and
maintenance of web sites and applications. In this paper, we present the work we have done to find out
which web metrics are considered useful by web developers and maintainers. This study has been
performed on the basis of the 385 web metrics classified in WQM, a Web Quality Model defined in a
previous work, using as validation tool, a survey made by professionals of web technologies. As a result, we
have found out that the most weighted metrics were related to usability. That means that web professionals
give more importance to the user of metrics than to their own effort.
1 INTRODUCTION
The spectacular development of the web has led to
an increasing importance of related technologies in
the functioning of organizations as well as in
people’s lives. It is compulsory that developed
products, both complex applications or simple web
sites, satisfy a minimum quality standard (Cutter
Consortium, 2000).
In the field of web metrics, a large research effort
has b
een made with proposals from very diverse
perspectives.
In spite of the fact that some of the proposed
m
etrics have not been formally defined or
theoretically or empirically validated, fortunately, in
the last years, the tendency is changing and
justification, formalization and validation are also
taken into account (Abrahão et al., 2003).
However, most of the work is academic and
doe
sn’t take into account industrial concerns. There
are some exceptions. Among them, we can cite the
works (Reifer, 2000, 2002) and (Mendes et al., 2003,
2005).
To eliminate the gap between practical
application and academ
ic world, it is necessary to
better involve the different actors related to web
technologies. To do so, it is essential to know the
perception of web metrics that these actors have.
With this objective, we have performed a survey
am
ong web technologies professionals to select the
metrics that are considered interesting or useful by
them. Once we have the set of metrics, we could
measure the web sites or the web applications and
obtain corresponding quality indicators.
Our starting point is to consider the metrics that
we collected
from the literature and were used in our
previous study (Calero et al., 2005) in which we
classified a total of 385 web metrics, using the
WQM quality model.
In the next section, we will expose the criteria
fo
llowed in the metrics selection that we have used
in our survey. In the third section, we will present
the survey, its results and conclusions. Finally, in the
fourth section, future work will be stated.
2 INTERNAL METRIC
SELECTION
As it is not possible to prepare a survey including
the 385 metrics classified in WQM, we performed a
238
Ruiz J., Calero C. and Piattini M. (2006).
WEB METRICS SELECTION THROUGH A PRACTITIONERS’ SURVEY.
In Proceedings of the First International Conference on Software and Data Technologies, pages 238-244
DOI: 10.5220/0001320502380244
Copyright
c
SciTePress
first selection with the objective of restricting them
into a manageable and representative set of web
metrics to be included in our survey.
2.1 Selection Criteria
The selection was made taking into account the
following considerations:
a) The number of metrics must be as limited as
possible.
b) The selection must cover the different
perspectives to be considered.
c) The most relevant works must be examined in a
detailed way, especially those having an
experimental component.
d) Works based on a concrete methodology must
not be refused at the beginning but we will have
to bear in mind the possibility of an easy
generalization.
e) Given that many aspects such as usability have a
great number of metrics, we will have to make an
even bigger synthesis effort than with other
aspects in which it is clear the lack of metrics.
f) It is not necessary that the selected number of
metrics must be proportional to the number of
collected metrics per aspect.
According to these considerations, we will
establish the following criteria for the selection
process:
1. To select those metrics that are proposed in
several works.
2. To select those metrics that represent simple
concepts.
3. To avoid duplicities, eliminating as much as
possible metrics that could be assimilated into
others, with respect to meaning, even not
representing the same concept.
4. To eliminate metrics coming from the
specialization of other metrics. Although they
can allow us to measure certain characteristics in
a more precise way, they can also made us lose a
more general vision.
5. To incorporate some metrics that are not very
common with the purpose of introducing
variability.
6. To incorporate metrics specific for a
methodology but able to be adapted to others.
Furthermore, if at any time there is a
contradiction between the criteria, we will prioritize
the simplest one.
2.2 Metrics Selection
As we have already indicated, our starting point has
been the 385 metrics.
According to criterion 1, we have a set of metrics
proposed by a large number of authors and that have
also a very simple meaning (criterion 2) such as
Number of Web Pages, Depth, Breadth, Number IN
Links, Total Number of Links, Number of Broken
Links, %Broken Links, Total Number of Images,
Images per Page, Images with ALT Text, all of them
with respect to the Website or Web Application, and
Download Time (of a page), Links of a Page and
Images of a Page.
We have also included others such as
Compactness, Stratum, and Cyclomatic Complexity,
based on criterion 5.
Following criteria 1, 2 and 5, we have included
the following: Quick Access Pages, Site Map,
Global Help, Scoped Search, Stability, Link Colour
Style Uniformity, Global Style Uniformity, Foreign
Language Support, Contact Address. And we have
selected other generic metrics (criteria 2) like
Suitable Information and Updated Information.
Concerning usability, as most of metrics are the
result of a specialization (criteria 4), we have
extracted the following (remember that other metrics
have been already included from other works):
Display Colour Count, Text Positioning Count, Text
Cluster Count, Font Count and Reading Complexity,
all of them with respect to a web page.
Regarding works related to the development and
maintenance of web applications, we have selected
(we have not included those included above),
following criterion 2: Media Count, Program Count,
Total Page Allocation, Total Media Allocation,
Total Code Length, Page Allocation, Media
Duration, Media Allocation, Code Length (LOC),
Code Comment Length, Reused Media Count,
Reused Program Count, Total Reused Media
Allocation, Total Reused Code Length, Reused Code
Length, Reused Comment Length, Total Page
Complexity, Page Complexity, Audio Complexity,
Video Complexity, Animation Complexity, Scanned
Image Complexity, Total Effort (Design&Auth),
Total Page Effort, Total Media Effort, Program
Effort, Experience, and Tool Type. And others like
Total Number Flash Animations, Total Number of
Icons/Buttons, Average Length Audio Clips, Average
Length Video Clips, Reused Web Pages, and Reused
Docs.
We include, according to criterion 6, (all of them
with respect to the web application): Web Building
Blocks, Number of COTS Components, Number of
WEB METRICS SELECTION THROUGH A PRACTITIONERS’ SURVEY
239
Object or Application Points, Number of XML,
SGML, HTML and Query Language Lines, Number
of Web Components, Number of Scripts (Visual
Language, Audio, Motion) and Number of Web
Objects. And based on criteria 5, we have taken the
metric Peak Staff.
Besides, due to the application of criterion 4, we
take into account the model efforts and the total
effort: Total Design Effort, Information Effort,
Navigation Effort and Presentation Effort.
Following criterion 2, we select Server Scripts,
Client Scripts, Web Page Scripts, Web Page
WebObjects, Total Languages and Page Languages.
We have not considered other metrics that are
not relevant as compared to the selected ones
because they mean an excessive specialization
(criterion 4).
With this selection we have obtained 85 metrics,
to be included in the survey (see appendix).
3 SURVEY
In this section, we will deal with aspects related to
the survey, its objective, design, obtained results and
its discussion.
3.1 Definition of Objectives
We have focused on the following objectives:
To determine the importance given by web
professionals to the considered web metrics.
To study the impact of participant experience on
the importance of metrics.
To identify other aspects not taken into account
in our work and that are considered important by
web practitioners.
To identify the concordances/discordances with
the metrics proposed by researchers in the
literature.
3.2 Survey Participants
An important aspect to be considered is who the
survey target since any community has its own
characteristics. For our purposes, our survey is
addressed to practitioners involved in tasks of
developing or maintaining applications and web
systems with diverse degree of experience.
Thus, the technical concepts should not represent
any problem. However, to fulfil our purpose, we
have to take into account other aspects. For example,
if the survey comes from the academic field can be
seen by web technologies professionals as it does
not fulfil their needs and they can refuse to fill it out.
Subjects were not involved only in a passive way. In
addition to theirs answers we tried to involve them
in the project by soliciting suggestions from them.
For this survey our objective population is the
web professionals. For the sample, we have
considered professionals that previously we had
maintained some contact in the past (or with their
companies). Choosing them to conserve the diversity
in the applications developed (scope of work), its
experience degree, and the companies for whom
work.
3.3 Survey Design
To fulfil the fixed objectives, we have structured our
survey into three parts:
A. Data of the Subject.
B. Web Metrics.
C. Suggestions.
We have to take into consideration that the
survey design is conditioned by the high number of
metrics to be included in it. Answering a survey with
questions about 85 metrics carries out certain
reticences regarding the necessary time to fill it out.
Now, we will deal with each part separately.
3.3.1 PART A: Data of the Subject
There are a great variety of web professionals
depending on their experience, their job and the
technologies that they use. For these reasons, and
following the recommendations of (Pfleeger and
Kitchenham, 2001) and (Kitchenham and Pfleeger,
2002a-d, 2003) we have included generic questions
about personal data. Thus, part A, Data of the
Subject, is composed of three questions:
1) Job (Developer, Maintenance Manager, Others)
2) Years of Experience
3) Category of the developed product:
a) Web site with static pages
b) Web sites with dynamic generation of pages,
working with jsp, php, asp, within a
centralized environment (e. g. applications
for small or medium size enterprise)
c) Web sites using Content Management
Systems (such as CMS of Microsoft, Zope,
Tipo3,…) in a distributed environment (e.g.
applications for a corporation)
ICSOFT 2006 - INTERNATIONAL CONFERENCE ON SOFTWARE AND DATA TECHNOLOGIES
240
3.3.2 PART B: Web Metrics
To make part B as simple as possible, we have
decided to use close questions, one for each metric,
quantifying the importance of each metric by using a
Likert scale with an interval from 0 (not important)
to 9 (very important).
To avoid fatigue and motivation loss we decided
to start the questioner by simple metrics.
For providing a common background to the
subjects, we included in the documentation a
minitutorial about the metrics of the survey.
3.3.3 PART C: Suggestions
This part has two open questions. The first is to
include suggestions of other metrics that subjects
consider interesting, and the second is to include
suggestions about the survey. The objective of this
part is, on the one hand, to detect metrics that we
have not considered, and on the other hand, to
validate our survey.
3.4 Results
The survey was sent to the subjects by personalized
email, avoiding as much as possible to give the
impression of being like a circular to avoid the
rejection rate. The survey could be sent back once
filled out in electronic or paper format.
66 surveys were sent and we obtained 42
answers (63.6%), during the ten days deadline.
Two participants classified themselves as web
users and the rest as professionals with different
degrees of experience and different development
environment. Then, the sample of web technologies
is formed by 40 subjects. From them, we have
centred our study as follows.
In the tables 1-4, we show the obtained results
for the different subjects categories. In each one of
the tables, column Score shows the arithmetic
average given by subjects of each considered group,
and Deviat. (or Dev) its standard deviation. As we
have noted above, each metric has been scored into a
scale from 0 (not important) to 9 (very important).
The interpretation of the results shown here will be
analysed in the following section.
For the set of the 40 subjects, the most accepted
metrics are shown in table 1. For those related to
technologies of static web pages (Category a, with
only 6 subjects), we obtain table 2 (we have
included this table in spite of the reduced sample
size). Regarding the most valued metrics for
categories b and c of subjects (34 subjects), we
obtain table 3.
In the table of the appendix, we can see the
complete relation of all the metrics studied in the
survey ordered according the importance given by
these 34 subjects of group b-c. Column Score is the
average score by the metrics, and Dev. its standard
deviation, for each of the group considered.
Table 1: Metrics rank for all the subjects (40 subjects).
Metric Score Deviat.
Updated Information 8.35 1.05
Suitable Information 8.13 1.26
D
ownload Time 8.05 1.11
Global Style Uniformity 7.88 1.11
Scoped Search 7.65 1.69
L
ink Colour Style Uniformity 7.58 1.32
Navigation Effor
t
7.55 1.50
I
nformation Effor
t
7.48 1.55
Total Effort (Design) 7.40 1.58
Presentation Effor
t
7.38 1.56
D
evelope
r
’s Experience 7.38 1.58
Quick Access Pages 7.38 1.50
Table 2: Metric rank for static web page developers.
Metric Score Dev
Suitable Information 8.33 0.82
Updated Information 8.33 0.82
F
oreign Language Suppor
t
8.00 0.89
% Broken Links 7.83 1.60
Global Style Uniformity 7.6
1.03
L
ink Colour Style Uniformity 7.50 0.84
Number of IN Links 7.1
0.98
Number of Broken Links 7.1
1.83
D
ownload Time 7.1
1.4
Global Help 7.00 1.10
Contact Address (e-mail, phone, mail) 7.00 0.63
Scoped Search 7.00 1.10
Table 3: Metric rank for b-c subjects category.
Metric Score Deviat.
Updated Information 8.35 1.10
D
ownload Time 8.21 0.98
Suitable Information 8.09 1.33
Global Style Uniformity 7.91 1.14
Scoped Search 7.76 1.76
Navigation Effor
t
7.74 1.26
I
nformation Effor
t
7.68 1.36
Total Effort (Design) 7.62 1.30
L
ink Colour Style Uniformity 7.59 1.40
Presentation Effor
t
7.56 1.33
D
eveloper’s Experience 7.53 1.54
WEB METRICS SELECTION THROUGH A PRACTITIONERS’ SURVEY
241
We have divided the group b-c in two subgroups:
the first one is formed by subjects with at least three
years’ experience (19 subjects), and, the other one is
formed by those subjects with less than three years’
experience (15 subjects). In the rest of the section
we will refer to them as subgroups I) and II),
respectively.
We have decided to perform this division since
we think that from three year’s experience there is a
qualitative leap in developer maturity as well as in
their knowledge of the technologies they work with.
For the sake of clarity, in table 4, we have
extracted the results corresponding to the best scored
metrics (score upon 7.50) according to subgroup I
(three or more years experience). As we can see
results do not differ very much with respect to those
obtained by complete category b-c (see table 3).
Table 4: Metrics rank for practitioners with 3 or more
years of experience in categories b-c.
Metric Score Deviat.
Updated Information 8.37 1.12
Download Time 8.32 0.95
Suitable Information 8.21 1.18
Scoped Search 8.16 1.01
Total Effort (Design) 8.11 0.94
Quick Access Pages 8.05 0.97
Information Effort 7.95 1.13
Total Effort (Design&Auth) 7.95 1.03
Global Style Uniformity 7.89 0.94
Navigation Effort 7.89 1.20
Presentation Effort 7.79 1.13
Number of Broken Links 7.79 1.96
Developer’s Experience 7.74 1.05
Page Allocation 7.53 2.06
Program Effort 7.53 1.47
Total Page Effort 7.53 1.43
Furthermore, if we compare the 22 most
highlighted metrics by subgroups I and II, we obtain
that there are more or less the same metrics except
for only four metrics for each subgroup that they do
not appear until positions 16
th
and 15
th
respectively
(see appendix).
3.5 Discussion and Interpretation of
Results
We notice that considerations about table 2 –
category a), static web page developers– will be only
indications, because of the reduced sample size.
The first conclusion we can extract is that
Usability is very important. This result was
foreseeable taking into account the importance of
usability in Web Applications (Calero et al., 2005).
Information quality is very important, Suitable
Information and Updated Information. We also note
the coincidence of groups a) and b-c) in other four
metrics among the most valued also related to
usability (Download Time, Scoped Search, Global
Style Uniformity, Link Colour Style Uniformity).
The rest of the metrics in table 3 are related to
effort (Information Effort, Navigation Effort, Total
Effort, Presentation Effort and Developer’s
Experience). It is paradoxical that developers
prioritize the user vs. their effort.
In opposition to the general perception, our
survey shows that importance granted in Literature
to the number of pages and the number of images
does not correspond to the perception that
developers have of them.
Nevertheless, we do not mean that the number of
pages is not important, since precisely the access to
information is made from a page (or a data entry if it
is carried out by other system). But these access
pages would be the important ones and not those
dynamically generated because the most important
aspects are the programs that generate pages, the
information they contain, how information is
presented, and not the number of such pages that can
be generated.
A similar reasoning can be made for the number
of images. In a small website (and normally static),
possibly image processing would be manual. But in
a system with thousands of items, they would be
provided in a digitalized format, probably in a
database. Consequently, system complexity should
be measured according to programs that use the
database, not to the database size.
As we have already mentioned, to achieve the
last cited objective, in section 3.1, we have
incorporated into the survey, a third section of
suggestions not only regarding metrics but also the
survey itself.
With relation to the suggestion of metrics, we
have found that almost all are also related to
Usability and in particular, to Accessibility and
adaptation to standards, compatibility with
navigators and, in a lower degree, others related to
performance and security.
3.6 Conclusions
In summary, the main conclusions we can extract
from the survey are:
¾ Developers prioritize usability instead of their
effort. By this, it is convenient to have tools that
ICSOFT 2006 - INTERNATIONAL CONFERENCE ON SOFTWARE AND DATA TECHNOLOGIES
242
from the first stages of development provide an
estimation of product usability.
¾ Some metrics frequently used in Literature, have
a relative importance for developers (e.g. number
of pages and number of images). This is because
the developed products are complex, with the use
of dynamic generation of pages, and the use of
Content Management Systems. There is a
necessity of metrics and frameworks for that.
4 FUTURE WORK
As we said in the introduction, our work has the
purpose of creating web applications quality
indicators. We consider it essential to take into
account the vision of quality of web professionals.
To do so, we have divided our task into two stages,
the first stage he consists in having a first
approximation to metrics considered relevant by
developers and sites and web applications
administrators. As a result of this, we have obtained
that the best valuated metrics are Updated
Information, Suitable Information, Download Time,
Global Style Uniformity and Scoped Search.
The second stage consists of determining the
importance of each metric with respect to each
quality characteristic and each phase of the life cycle
process. Therefore, we must obtain the metric
weights with the purpose of obtaining quality
indicators of a site or a web application.
However, before starting the second stage, we
aim at carrying out the survey again using a different
group of subjects to check the validity of the
obtained results. The indicators obtained in our work
must be able to be incorporated into web
development methodologies.
Other aspect to be considered (considering the
importance given to usability) is the incorporation of
metrics for estimating the end product usability
during the development. The same happens with
accessibility.
ACKNOWLEDGEMENTS
We would like to thank all participants in this survey
for their time and above all, for their suggestions and
especially to those that not only indicated
suggestions in the survey but also came to talk to the
authors about new approaches in our work. Thank
you all.
This work is part of the CALIPO project (TIC
2003-07804-C05-03) and the CALIPSO network
(TIN2005-24055-E) supported by the Spanish
Ministerio de Educación y Ciencia and by the
DIMENSIONS project (PBC-05-012-1) supported
by FEDER and Junta de Comunidades de Castilla-
La Mancha.
REFERENCES
Abrahão, S., Condori-Fernandez, N., Olsina, L., Pastor,
O., 2003. Defining and Validating Metrics for
Navigational Models, in METRICS'03, 9
th
Intern.
Software Metrics Symposium, IEEE. pp. 200-210.
Calero, C., Ruiz, J., Piattini, M., 2005. Classifying Web
Metrics Using the Web Quality Model. Information
Online Review. Vol. 29, n.3, June 2005. pp. 227-248.
Cutter Consortium, 2000. Poor Project Management –
Problem of E-Projects. October 2000,
http://www.cutter.com/press/001019.html
Kitchenham, B., Pfleeger S.L., 2002a. Principles of survey
research Part 2: Designing a Survey. SIGSOFT Soft.
Eng. Notes, ACM Press, 2002. 27(1) pp. 18-20.
Kitchenham, B., Pfleeger S.L., 2002b. Principles of survey
research: part 3: constructing a survey instrument.
SIGSOFT Soft. Eng. Notes, ACM Press, 2002. 27(2)
pp. 20-24.
Kitchenham, B., Pfleeger S.L., 2002c. Principles of survey
research part 4: questionnaire evaluation. SIGSOFT
Soft. Eng. Notes, ACM Press, 2002. 27(3) pp. 20-23.
Kitchenham, B., Pfleeger S.L. , 2002d. Principles of
survey research: part 5: populations and samples.
SIGSOFT Soft. Eng. Notes, ACM Press, 2002. 27(5)
pp. 17-20.
Kitchenham, B., Pfleeger S.L., 2003. Principles of survey
research part 6: data analysis. SIGSOFT Soft. Eng.
Notes, ACM Press, 2003. 28(2) pp. 24-27.
Mendes, E., Mosley, N., Counsell, S. 2003. Early Web
Size Measures and Effort Prediction for Web
Costimation, Proc. 9
th
Int. Metrics Symp.2003
(METRICS’03). pp. 18-29.
Mendes, E., Mosley, N., Counsell, S. 2005. Investigating
Web Size Metrics for Early Web Cost Estimation,
Journal of Systems and Software, Vol. 77, No. 2,
August 2005, pp. 157-172
Pfleeger, S.L., Kitchenham, B., 2001. Principles of survey
research: part 1: turning lemons into lemonade.
SIGSOFT Soft. Eng. Notes, ACM Press, 2001. 26(6)
pp. 16-18.
Reifer, D., 2000. Web Development: Estimating Quick-to-
Market Software. IEEE Software, Nov-Dec. 2000. pp.
57-64.
Reifer, D., 2002. Estimating Web Development Costs:
There are Differences, Crosstalk, June 2002, pp. 13-
17.
WEB METRICS SELECTION THROUGH A PRACTITIONERS’ SURVEY
243
APPENDIX
Metric
Score in group b-c
(34 subjects)
Score in group b-c
Subgroup
Experience>=3 years
(19 subjects)
Score in group b-c
Subgroup
Experience<3 years
(15 subject)
Score in group a
(6 subjects)
All Subjects
(40 subjects)
A
ver. Dev.
A
ver. Dev.
A
ver. Dev.
A
ver. Dev.
A
ver. Dev.
Updated
Inf
o
rm
a
t
i
o
n
8
.
35
1.1
0
8
.
3
71.12
8
.
33
1
,
11
8,33
0,8
2
8,35
1
,05
D
o
wnl
oad
Tim
e
8
.21
0
.
98
8
.
3
2
0
.
95
8
.
0
71
,03
7
,
17 1
,
47
8,05
1
,
11
Su
i
tab
l
e
Inf
o
rm
at
i
o
n
8
.
09
1.
33
8
.21 1.1
8
7.
93
1
,53
8,33
0,8
2
8,
1
3
1
,
2
6
G
l
oba
l
Sty
l
e
U
nif
o
rmi
ty
7.
9
11.14 7.
89
0
.
9
47.
93
1
,39
7
,6
7 1
,03
7
,88
1
,
11
Scoped
Sea
r
c
h 7.7
6
1.7
6
8
.1
6
1.
0
17.272
,3
47
,00
1
,
1
0
7
,65
1
,69
N
a
vi
gat
i
o
n Eff
o
r
t
7.74 1.2
6
7.
89
1.2
0
7.
53
1
,36
6,50
2
,35
7
,55
1
,50
Inf
o
rm
at
i
o
n Eff
o
r
t
7.
68
1.
36
7.
95
1.1
3
7.
33
1
,59
6,33
2
,
1
6
7
,
4
8
1
,55
T
ota
l Eff
o
r
t
(
D
es
i
g
n
)
7.
6
21.
30
8
.11
0
.
9
47.
00
1
,
4
6
6,
17 2
,
4
8
7
,
4
0
1
,58
Link
Co
l
ou
r
Sty
l
e
U
nif
o
rmi
ty
7.
59
1.4
0
7.47 1.2
6
7.7
3
1
,58
7
,50
0,8
4 7
,58
1
,3
2
Pr
ese
n
tat
i
o
n Eff
o
r
t
7.
56
1.
33
7.7
9
1.1
3
7.27 1
,53
6,33
2
,
42 7
,38
1
,56
D
e
v
e
l
ope
r'
s
Ex
pe
ri
e
n
ce
7.
53
1.
5
47.741.
05
7.27 2
,0
2
6,50
1
,6
4 7
,38
1
,58
Qu
i
c
k A
ccess
P
ages
7.47 1.
56
8
.
05
0
.
9
7
6
.7
3
1
,8
7
6,83
0,98
7
,38
1
,50
P
age
All
ocat
i
o
n 7.44 2.
08
7.
53
2.
06
7.
33
2
,
1
6
6,00
2
,
2
8
7
,
2
3
2
,
14
T
ota
l Eff
o
r
t
(
D
es
i
g
n
&
A
ut
h
)
7.41 1.
3
77.
95
1.
03
6
.7
3
1
,
4
9
5,83
3,06
7
,
1
8
1
,
77
N
u
m
be
r
o
f Br
o
k
e
n Link
s
7.
35
2.
09
7.7
9
1.
96
6
.
80
2
,
1
8
7
,
17 1
,83
7
,33
2
,03
G
l
oba
l H
e
l
p
7.2
6
1.
36
7.2
6
1.2
8
7.27 1
,
4
9
7
,00
1
,
1
0
7
,
2
3
1
,3
1
Pr
og
r
a
m Eff
o
r
t
7.1
8
1.
38
7.
53
1.47
6
.7
3
1
,
1
6
5,
17 2
,
4
0
6,88
1
,
7
0
T
ota
l P
age
Eff
o
r
t
7.1
5
1.4
8
7.
53
1.4
3
6
.
6
71
,
4
5
5,6
7 2
,88
6,93
1
,
7
9
%
Br
o
k
e
n Link
s
7.
00
2.1
5
7.21 2.2
3
6
.7
3
2
,09
7
,83
1
,60
7
,
1
3
2
,08
F
o
r
e
i
g
n L
a
n
guage
Suppo
r
t
6
.
9
12.27 7.
00
2.1
3
6
.
80
2
,5
1
8,00
0,89
7
,08
2
,
1
5
R
ead
in
g
Co
m
p
l
e
xi
ty
6
.
88
1.
56
7.44 1.
0
4
6
.2
0
1
,8
2
6,83
2
,56
6,8
7 1
,
7
0
N
u
m
b
.
o
f XML
,
SG
ML
,
HTML
a
n
d
Que
r
y
L
a
n
guage
Lin
es
6
.
85
1.2
6
6
.74 1.
05
7.
00
1
,5
1
6,00
1
,6
7
6,
7
3
1
,3
4
St
r
atu
m
6
.
8
22.
0
1
6
.
9
42.2
9
6
.
6
71
,68
6,00
1
,
7
9
6,69
1
,98
T
ota
l M
ed
i
a
Eff
o
r
t
6
.74 1.
99
7.
3
21.
5
7
6
.
00
2
,
27 4
,33
2
,58
6,38
2
,
2
3
S
i
te
M
ap
6
.
65
1.
86
6
.
95
1.
68
6
.27 2
,05
6,50
0,8
4
6,63
1
,
7
3
N
u
m
be
r
o
f W
eb
P
age
Sc
ri
pts
6
.
6
21.41
6
.
63
1.
30
6
.
60
1
,59
6,50
1
,5
2
6,60
1
,
41
T
oo
l T
ype
6
.
6
21.
9
1
6
.
53
2.
06
6
.7
3
1
,
7
5
5,83
2
,
14
6,50
1
,93
Im
pu
ri
ty
Tr
ee
6
.
6
11.
80
6
.
56
2.
09
6
.
6
71
,
4
5
6,83
1
,
47
6,6
4 1
,
74
N
u
m
be
r
o
f R
eused
W
eb
P
ages
6
.
56
1.
93
6
.
58
1.4
6
6
.
53
2
,
4
5
6,6
7 1
,
21
6,58
1
,8
2
Co
n
tact
A
dd
r
ess
(e
-m
a
il
,
p
h
o
n
e,
m
a
il
)
6
.
53
2.
33
7.21 1.
93
5
.
6
72
,55
7
,00
0,63
6,60
2
,
1
6
N
u
m
be
r
o
f
Se
rv
e
r
Sc
ri
pts
6
.47 1.
6
7
6
.21 1.
8
4
6
.
80
1
,
42
6,50
1
,5
2
6,
4
8
1
,63
N
u
m
be
r
o
f
C
li
e
n
t
Sc
ri
pts
6
.47 1.71
6
.
53
1.2
6
6
.4
0
2
,
2
0
6,83
1
,
17
6,53
1
,63
N
u
m
be
r
o
f W
eb
Objects
6
.4
5
1.
66
6
.
39
1.
33
6
.
53
2
,03
6,33
1
,
21
6,
44 1
,59
T
ota
l R
eused
Code
L
e
n
gt
h
6
.44 2.
0
2
6
.
58
1.
68
6
.27 2
,
4
3
6,
17 1
,
72
6,
4
0
1
,96
R
eused
Code
L
e
n
gt
h
6
.41 1.
99
6
.42 1.77
6
.4
0
2
,
2
9
5,83
1
,
47
6,33
1
,9
1
T
ota
l
Code
L
e
n
gt
h
6
.
35
1.74
6
.
63
1.
5
7
6
.
00
1
,93
6,
17 1
,
17
6
,33
1
,65
Co
m
pact
n
ess
6
.
30
1.47
6
.
39
1.
69
6
.2
0
1
,
21
6,6
7 1
,
21
6,36
1
,
42
R
eused
Pr
og
r
a
m
Cou
n
t
6
.2
9
1.
90
6
.2
6
1.
59
6
.
33
2
,
2
9
6,33
1
,86
6,30
1
,8
7
N
u
m
be
r
o
f
Object
o
r A
pp
li
cat
i
o
n P
o
in
ts
6
.2
9
1.4
3
6
.2
6
1.
33
6
.
33
1
,59
6,6
7 1
,03
6,35
1
,3
7
W
eb
B
u
il
d
in
g
Bl
oc
k
s
6
.2
6
2.14
6
.21 2.
0
7
6
.
33
2
,
2
9
5,50
0,8
4
6,
1
5
2
,0
1
N
u
m
be
r
o
f
Sc
ri
pts
(
Vi
sua
l L
a
n
guage,
A
ud
i
o,
M
ot
i
o
n
)
6
.24 1.74
6
.2
6
1.
8
2
6
.2
0
1
,
7
0
6,
17 1
,60
6,
2
3
1
,
7
0
N
u
m
be
r
o
f W
eb
Co
m
po
n
e
n
ts
6
.24 1.
3
7
6
.
3
21.1
6
6
.1
3
1
,6
4
6,50
1
,05
6,
2
8
1
,3
2
D
ept
h
6
.1
8
1.
8
7
6
.47 1.
58
5
.
80
2
,
1
8
6,83
1
,60
6,
2
8
1
,83
T
ota
l P
age
All
ocat
i
o
n
6
.1
8
2.
38
6
.2
6
2.2
8
6
.
0
72
,58
5,33
3,0
1
6,05
2
,
4
6
Code
L
e
n
gt
h
(
L
OC)
6
.12 1.
85
6
.
3
21.
80
5
.
8
71
,96
5,83
1
,
47
6,08
1
,
7
9
T
ota
l N
u
m
be
r
o
f I
co
n
s/
B
utto
n
s
6
.
09
1.
83
6
.
3
71.
6
4
5
.7
3
2
,05
5,6
7 1
,5
1
6,03
1
,
7
8
N
u
m
b
e
r
o
f
CO
T
S
Co
m
po
n
e
n
ts
6
.
09
1.
6
4
6
.21
0
.
98
5
.
93
2
,
2
5
6,33
0,5
2
6,
1
3
1
,5
2
W
eb
P
age
W
ebObjects
6
.
09
1.
8
1
6
.11 1.4
5
6
.
0
72
,
22
6,50
1
,05
6,
1
5
1
,
71
Pr
og
r
a
m
Cou
n
t
5
.
9
71.
96
6
.
05
1.
5
4
5
.
8
72
,
4
5
5,50
1
,9
7
5,90
1
,95
Di
sp
l
ay
Co
l
ou
r
Cou
n
t
5
.
9
42.2
0
6
.
00
2.
36
5
.
8
72
,0
7
6,50
1
,5
2
6,03
2
,
11
Br
eat
h
5
.
85
1.
8
4
5
.
95
1.
8
4
5
.7
3
1
,9
1
6,
17 1
,9
4
5,90
1
,8
4
P
ea
k
Sta
ff
5
.
85
2.
09
5
.
8
42.
3
4
5
.
8
71
,8
1
5,83
1
,83
5,85
2
,03
T
ota
l M
ed
i
a
All
ocat
i
o
n
5
.
8
22.47
5
.
58
2.22
6
.1
3
2
,80
6,
17 2
,
14
5,88
2
,
4
0
Code
Co
mm
e
n
t
L
e
n
gt
h
5
.7
6
2.1
3
6
.
3
21.
63
5
.
0
72
,5
2
6,33
1
,5
1
5,85
2
,05
T
e
x
t
P
os
i
t
i
o
nin
g
Cou
n
t
5
.7
6
2.
35
5
.
58
2.4
8
6
.
00
2
,
24
6,33
1
,63
5,85
2
,
2
5
N
u
m
be
r
o
f W
eb
P
ages
5
.
65
1.
9
4
5
.
89
2.
0
2
5
.
33
1
,8
4
6,33
1
,63
5,
7
5
1
,89
M
ed
i
a
All
ocat
i
o
n
5
.
65
2.41
5
.47 2.
3
2
5
.
8
72
,59
5,6
7 1
,9
7
5,65
2
,33
F
o
n
t
Cou
n
t
5
.
59
1.
9
7
6
.
05
2.
0
4
5
.
00
1
,
77
5,6
7 1
,5
1
5,60
1
,89
T
e
x
t
C
l
uste
r
Cou
n
t
o
f
a
P
age
5
.
58
2.22
5
.
50
2.
53
5
.
6
71
,88
6,
17
0,98
5,6
7 2
,08
Link
s
o
f
a
P
age
5
.
53
2.
09
5
.2
6
2.1
8
5
.
8
72
,00
6,00
1
,
7
9
5,60
2
,0
4
N
u
m
be
r
o
f T
ota
l Link
s
5
.47 1.
9
4
5
.47 2.
06
5
.47 1
,85
6,
17 1
,
72
5,58
1
,9
1
N
u
m
be
r IN Link
s
5
.44 2.
06
5
.
3
72.14
5
.
53
2
,03
7
,
17
0,98
5,
7
0
2
,03
Co
nn
ect
ivi
ty
D
e
n
s
i
ty
5
.44 2.27
5
.
3
72.
5
2
5
.
53
2
,00
6,
17 1
,
17
5,55
2
,
1
5
T
ota
l L
a
n
guages
5
.41 1.
9
7
5
.
3
22.
3
1
5
.
53
1
,5
14
,83
0,98
5,33
1
,86
Im
ages
wi
t
h ALT T
e
x
t
5
.
35
2.44
5
.
8
42.
36
4.7
3
2
,
4
6
5,50
1
,8
7
5,38
2
,3
4
Im
ages
pe
r P
age
5
.2
9
2.14
5
.
89
1.
85
4.
53
2
,
2
9
6,00
0,63
5,
4
0
2
,00
T
ota
l N
u
m
be
r
o
f Im
ages
5
.21 2.2
5
5
.
53
2.17 4.
80
2
,3
74
,6
7 1
,3
7
5,
1
3
2
,
14
M
ed
i
a
D
u
r
at
i
o
n
5
.12 2.4
8
4.74 2.1
8
5
.
60
2
,8
24
,
17 1
,83
4
,98
2
,
4
0
N
u
m
be
r
o
f P
age
L
a
n
guages
5
.
06
1.
9
74.
89
2.1
8
5
.27 1
,
71
5,
17 1
,
17
5,08
1
,86
Im
ages
o
f
a
P
age
5
.
03
2.1
0
5
.
3
22.
00
4.
6
72
,
2
3
4
,83
0,98
5,00
1
,96
R
eused
M
ed
i
a
Cou
n
t
4.
9
42.
5
14.7
9
2.
3
7
5
.1
3
2
,
7
5
6,33
1
,86
5,
1
5
2
,
4
6
Cyc
l
o
m
at
i
c
Co
m
p
l
e
xi
ty
4.
9
12.4
0
4.
89
2.
6
24.
93
2
,
1
9
6,6
7 1
,63
5,
1
8
2
,3
7
A
v
e
r
age
L
e
n
gt
h Vi
deo
C
li
ps
4.
9
12.
35
4.
53
2.12
5
.4
0
2
,6
14
,50
2
,
4
3
4
,85
2
,3
4
T
ota
l R
eused
M
ed
i
a
All
ocat
i
o
n 4.
9
12.
50
4.
89
2.4
9
4.
93
2
,60
6,00
2
,6
1
5,08
2
,5
2
T
ota
l N
u
m
be
r Fl
as
h Anim
at
i
o
n
s
4.
85
2.41
5
.2
6
2.4
0
4.
33
2
,
41
3,83
1
,83
4
,
7
0
2
,3
4
A
v
e
r
age
L
e
n
gt
h A
ud
i
o
C
li
ps
4.
85
2.
38
4.47 2.14
5
.
33
2
,6
44
,33
2
,
2
5
4
,
7
8
2
,3
4
A
nim
at
i
o
n
Co
m
p
l
e
xi
ty
4.
8
22.
3
44.
89
2.4
0
4.7
3
2
,3
44
,33
1
,
21 4
,
7
5
2
,
2
0
R
eused
D
ocs
4.7
9
2.
33
4.
95
2.1
5
4.
60
2
,6
1
6,
17 1
,
17
5,00
2
,
24
M
ed
i
a
Cou
n
t
4.71 2.1
8
5
.
05
2.
09
4.27 2
,
2
8
4
,6
7 1
,63
4
,
7
0
2
,09
P
age
Co
m
p
l
e
xi
ty
4.
65
2.1
0
4.
68
2.24 4.
60
1
,99
6,00
1
,6
7 4
,85
2
,08
Aud
i
o
Co
m
p
l
e
xi
ty
4.
53
2.2
5
4.
58
2.24 4.47 2
,33
4
,
17 1
,
17 4
,
4
8
2
,
11
Vi
deo
Co
m
p
l
e
xi
ty
4.
50
2.
36
4.
58
2.4
6
4.4
0
2
,3
24
,00
1
,
7
9
4
,
4
3
2
,
27
T
ota
l P
age
Co
m
p
l
e
xi
ty
4.
3
22.2
0
4.
58
2.
3
24.
00
2
,0
7
3,6
7 1
,3
7 4
,
2
3
2
,09
Sca
nn
ed
Im
age
Co
m
p
l
e
xi
ty
4.
09
2.
0
24.
3
21.
95
3
.
80
2
,
14
3,
17 1
,
47
3,95
1
,96
R
eused
Co
mm
e
n
t
L
e
n
gt
h
3
.
85
2.2
8
4.2
6
2.47
3
.
33
1
,99
3,83
2
,3
2
3,85
2
,
2
6
ICSOFT 2006 - INTERNATIONAL CONFERENCE ON SOFTWARE AND DATA TECHNOLOGIES
244