A Case of Adoption of 25000 Standards Family
Establishing Evaluation Requirements in the Audio-Visual Preservation Context
Isabella Biscoglio and Eda Marchetti
Institute of Information Science and Technologies “Alessandro Faedo” of the National Research Council, Pisa, Italy
Keywords: Software Quality, Requirements Elicitation, Digital Audio-Visual Media Preservation.
Abstract: The digital preservation want to guarantee accessible and usable over time digital audio-visual media
content, regardless of the challenges of media failure and technological change. For this aim, the current
technologies for digital audio-visual media preservation deal with complex technological, organizational,
economic and rights-related issues. Ensuring the development and use of high-quality software could be a
key factor for their success. The paper reports an experience matured inside the Presto4U project, with the
contribution of the TATE Gallery. Aim of the experiment was the identification of preservation needs and
corresponding preservation requirements in order to set up a quality evaluation process for used software
tools making easy the technology transfer of research results into digital preservation environment. Using
the framework of the international standards on software products quality, the evaluation requirements were
mapped on the characteristics and sub-characteristics of a quality model and a specification of high-level
software product quality evaluation plan has been obtained and presented.
1 INTRODUCTION
The long-term preservation of digital audio-visual
media presents a range of complex technological,
organizational, economic and rights-related issues,
which have been the subject of intensive research
over the past fifteen years at national, European and
international levels. Although good solutions are
emerging and there is a large body of expertise at a
few specialist centers, it is very difficult for the great
majority of media owners to gain access to advanced
audio-visual preservation technologies.
Considering the dynamic nature of the IT
industry, audiovideo media and research
technologies are subject to many changes in
hardware and software capabilities. Technology
cycles are short and therefore product lifetimes tend
to be short, producing decay and technological
obsolescence. These problems have an impact on
digital material preservation in terms of source and
process of preservation.
In this context one of the currently on-going
European project targeting the topic of audiovisual
preservation is Presto4U project - European
Technology for Digital Audiovisual Media
Preservation (www.prestocentre.org/4u). Supporting
an important technology transfer between research
and industry, the Presto4U project focuses research
efforts onto useful technological solutions, in order
to raise awareness of the needs of audiovisual
preservation and to improve the adoption of
audiovisual preservation research results, both by
service providers and media owners. Encouraging
the organization of the principal actors of
audiovisual media preservation (users, technology
vendors and service providers) in a set of
Communities of Practice (CoPs), the project aims to
develop a body of knowledge on the status of digital
preservation practice (problems and needs), to
identify useful research results and to promote their
adoption and implementation.
In particular, identifying a quality evaluation
process able to guarantee the development and use
of high-quality software both by technology and
service providers as well as media owners
constitutes the main objective. Thus a crucial work
of the project, and thus of this paper, is the
identification of the quality needs and the
identification of models for quality achievement.
Indeed these models will be useful to achieving the
established quality level and to make easy the
technology transfer of research results into digital
preservation environment. The ultimate purpose is
improving the commercial technologies
222
Biscoglio I. and Marchetti E..
A Case of Adoption of 25000 Standards Family - Establishing Evaluation Requirements in the Audio-Visual Preservation Context.
DOI: 10.5220/0004994502220233
In Proceedings of the 9th International Conference on Software Engineering and Applications (ICSOFT-EA-2014), pages 222-233
ISBN: 978-989-758-036-9
Copyright
c
2014 SCITEPRESS (Science and Technology Publications, Lda.)
development and promoting new technical
approaches to match the specific preservation needs
of each CoP.
On the other hand, software quality and its
evaluation are not new research topics. They have
been discussed for several years and from different
points of view (Krause, Freimut, Suryn, 2003,
Pfleeger 2009, Vliet, 2002). Achieving software
product quality could be summarized as achieving
the required quality through the definition and
implementation of quality requirements, the
measurement of appropriate quality characteristics
and the evaluation of the resulting quality. Also in
the context of standardization, quality is very well
defined and many international standards have been
published about software processes and products
quality (ISO 9126, 1991, ISO/IEC 9126 – 1, 2001,
ISO/IEC 14598, 1998, ISO/IEC 15504, 2004,
ISO/IEC FDIS 25000, 2005).
In the digital preservation context, one of the
most important reference model developed in a ISO
standard is Open Archival Information System
(OAIS) (ISO 14721, 2012). This model concerns
technical aspects of digital object’s life cycle, as
ingest, archival storage, data management and
access, and it also recommends metadata issues. It
mainly targets the preservation functional aspects
and it does not deal with non-functional quality
aspects. Consequently, this paper does not rely on
OAIS standard but it overviewed available standards
on software technologies from point of view of the
quality characteristics and their adherence to
preservation needs.
However to the best of our knowledge none of
the available quality standards and/or best practices
has been currently specialized and adapted for the
specific digital audio-visual preservation context.
Indeed all these documents and sources of
information constitute a very interesting base for the
digital preservation context, as models,
characteristics, measures and methods that they
present can be considered or readapted for
preservation purposes. Current and available audio-
visual technologies could be evaluated on the basis
of software quality standards and the results could
suggest important decisions about their adoption or
improvement. Nevertheless, for this aim it is
necessary a preliminary activity of preservation
requirement elicitation and a successive mapping of
the preservation needs on the characteristics and
sub-characteristics of a quality model on which these
requirements impact. Only later an evaluation
process could be implementable.
Regarding to the software quality evaluation
process, the paper reports an experience of
“establishing the evaluation requirements” (ISO/IEC
25040, 2010) matured inside the Presto4U project.
TATE Gallery (www.tate.org.uk) contributed to the
detection of some preservation needs in the context
of audio-visual collection.
With respect to specific quality issues for the
digital collections, following activities are carried
out:
the definition of preservation needs;
the specification of these needs in software
product quality evaluation requirements;
the identification of a software product quality
model;
the mapping between the requirements and
characteristics and sub-characteristics of the
detected quality model;
the refinement of a preliminary high level
software product quality evaluation plan,
according to selected quality model and with a
focus on the more perceived quality issues.
During the implementation of case study, both
the on-the-field practice and the research experience
of the involved partners have been exploited. The
obtained results are a specification of software
product quality evaluation requirements and a
specification of a preliminary high level software
product quality evaluation plan. In the perspective of
a software product quality evaluation process, these
results represent a necessary intermediate product,
for carry on the evaluation and obtain important
results (ISO/IEC 25040, 2010).
The paper is organized as in the following: the
Section 2 presents the motivation of this work. The
Section 3 introduces the topic of software quality
and an overview about the international standards
for software quality. In Section 4, the quality
evaluation procedure and the adopted quality model
are presented. In Section 5, the explorative case
study is shown, and a mapping between
requirements and characteristics was performed.
Discussion and conclusion close the paper.
2 MOTIVATION
The decay and the technological obsolescence of the
software products that are used in digital material
preservation context represent a serious problem in
terms of sources and process of preservation, content
management, digital archiving, etc.
Identifying a quality evaluation process able to
guarantee the development and use of high-quality
ACaseofAdoptionof25000StandardsFamily-EstablishingEvaluationRequirementsintheAudio-VisualPreservation
Context
223
software could help both technology and service
providers as well as media owners.
The identification of international standards for
the software products quality could be useful to limit
the above-cited risks of decay and the technological
obsolescence. Besides, involving multiple
stakeholders, the quality evaluation process could
make easy the technology transfer of research results
into digital preservation environment. Thus models,
characteristics, measures and methods that the
standards offer could be considered or readapted for
audio-visual preservation purposes.
However as discussed in Polillo (Polillo, 2012)
these standards only provide a conceptual
framework, and not a ready-to-use solutions usable
in every context. The necessity of adaptation and
revision becomes a pressing exigency, mainly when
the considered systems or products do not perfectly
fit with the characteristics and peculiarities of
classical software engineering
Besides, even when the standards are very
comprehensive and specific, they still show some
common weaknesses, which could have an
important impact on applicability of the standards
themselves (Azuma, 2011)
As an example in Rafa and Al-Quataish (Rafa E.
Al-Quataish, 2009) problems with (ISO/IEC 9126 –
1, 2001), not already completely solved in the new
SQuaRE version, have been highlighted.
In particular some weaknesses of the available
standards that have been identified by different
researchers concern over the years are (Olsina, Lew,
Dieser and Rivera 2012):
the terminology: it could be not fully aligned
with the classic measurement terminology in
software engineering (Abran, Al-Qutaish, and
Desharnais, 2005);
the metrics identification: the metrics provided
by the various standard could not be properly
aligned. For instance ISO 9126 metrics should
be classified and mapped according to those
proposed by the ISO 15939 (Abran, Al-
Qutaish, Desharnais, and Habra, 2005);
the results analysis: not all the standards
propose the same ranking for the considered
metrics. Thus possible conversions or
alignment of various ranking considered
should be defined. For instance a possible
conversion of the result of the ISO 9126
metrics from percentage values to ranked
values in terms of qualitative values should be
considered (Abran, Al-Qutaish and
Cuadrado-Gallego, 2006);
the same reference scale: wherever possible for
the same ranking the same reference scale
should be adopted so make easier finals
quality assessment value. For instance every
metrics of the ISO 9126 should be mapped to
the same reference scale (Koscianski and
Bracarense Costa, 1999);
the extension of the characteristics: each quality
standards should try to include or extend the
characteristics already defined in different
available standards so to have a more uniform
and complete coverage of quality
requirements. For instance the opportunity the
usability characteristic the ISO 9126 standard
could be extended to include more specific
sub-characteristics such as consistency,
simplicity, legibility, colour use, and help
(Chua and Dyson, 2004 and Zádor, Balla,
Trienekens, Kusters, 2008).
As highlighted in Bhatti et al. (Bhatti, Abdullah
and Gencel, 2009) about 80% of the measurement
programs were reported to fail to helping in decision
making or to delivering performance improvements.
According to Bundschuh et al. and Gopal et al.
(Bundschuh, Dekkers, 2008, Gopal, Krishnan,
Mukhopadhyay, Goldenson, 2002 and Hall and
Fenton, 1997), the risk of failure for the
measurement programs could be:
a misunderstanding of what is to be measured,
why and how it is to be measured;
an inadequate data collection;
a wrong interpretations of data;
the lack of trained and expert resources
required to dedicate to measurement;
the expensive costs for measurement programs;
the incorrect mapping of organization goals
with appropriate measures.
The criticalities underlined in the various quality
evaluation procedures in many cases make very
difficult the adoption of the standards into the
products development.
In spite of the limits of international standards
applicability, considering the quality as a limited
number of independent characteristics (ISO/IEC
9126 – 1, 2001) allows to evaluate the most
interesting for each Communities of Practices
characteristics and quality levels (“internal”,
“external” and “in-use”).
The ambition of this paper is to overcome some
of the above-mentioned weaknesses considering the
specific context of the audio-visual preservation.
In particular, for improving the common
understanding and decreasing as much as possible
the points of doubts, an alignment of the classical
ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications
224
terminology of the software engineering context
with the one that is specific for the preservation
context has been pursued. In order to reduce the gap
between general definition of the characteristics and
associated metrics defined by the ISO standards and
terminology and goals of the preservation
environments, a shared and informal glossary has
been defined preliminary.
As main objective of the project, and thus of this
paper, is identifying a quality evaluation process
able to guarantee the development and use of high-
quality software both by technology and service
providers as well as media owners, a crucial activity
is the identification of the quality needs and the
quality models for quality achievement. Therefore,
this work focuses on the audio-video preservation
needs and related requirements so to enhance the
quality evaluation procedure and the adoption of the
international standards.
In this paper, the experiment stops at this phase,
because, as above reported, the project is currently
on-going. As future work, the result of this
experiment, that is a high-level software product
quality evaluation plan, will be refined and
specialized with measures, techniques, criteria, etc.
and used by each CoP for evaluating their tools.
Since the standards applicability is expensive and
involves many resources requiring strong agreement
among stakeholder groups, in this phase a quality
evaluation plan has been specialized by means of an
accurate selection of measurable characteristics,
related to the organizational goals, in order to reduce
costs and efforts due to the evaluation of the quality.
3 BACKGROUND
In the field of software engineering, the evaluation
of software product quality is vital to both
acquisition and development of software that meets
quality requirements.
In the following, some of the most important
international standards will be presented.
For aim of completeness, some of the specific
terms used inside this paper, according to the
indications of many international standards, are
firstly reported. In particular:
Software quality, that is defined as “the totality
of features and characteristics of a software
product that bear on its ability to satisfy stated
or implied needs” (ISO 9126, 1991).
Software quality characteristics, that are
defined as “a set of attributes of a software
product by which its quality is described and
evaluated. A software quality characteristic
may be refined into multiple levels of sub-
characteristics” (ISO 9126, 1991).
Software quality metric, that is “a quantitative
scale and method, which can be used to
determine the value a feature takes for a
specific software product” (ISO 9126, 1991).
Assessment, that is defined as “an action of
applying specific documented assessment
criteria to a specific software module, package
or product for the purpose of determining
acceptance or release of the software module,
package or product” (ISO 9126, 1991).
Software quality assessment criteria that are
meant as “the set of defined and documented
rules and conditions which are used to decide
whether the total quality of a specific software
product is acceptable or not. The quality is
represented by the set of rated levels
associated with the software product” (ISO
9000-3,1991).
About software quality, the standardization
bodies that have mostly published are:
ISO (International Organization for
Standardization).
IEC (International Electrotechnical
Commission).
IEEE (Institute for Electrical and Electronic
Engineers).
The IEC cooperates closely with ISO and IEEE,
and the standards developed jointly with ISO carry
the acronym of both organizations. In the following
section some of the most important standards of the
above mentioned standardization body are briefly
introduced.
3.1 ISO/IEC Standards
Considering the ISO and IEC, two of the most
important joint standards developed for the software
quality evaluation have been: the ISO/IEC 9126
“Software engineering - Product Quality” with its
four versions (ISO/IEC 9126 – 1, 2001, ISO/IEC
9126 – 2, 2001, ISO/IEC 9126 – 3, 2001, ISO/IEC
9126 – 4, 2001) and the ISO/IEC 14598 “Software
engineering – Product Evaluation” (ISO/IEC 14598,
1998).
The principal merit of ISO/IEC 9126 standard
can be found in its attempt to reduce the product
quality concept to a limited number of independent
characteristics and to have developed the notion of
various levels of qualities (“internal”, “external” and
“in-use”). Nevertheless, it was not successful in
providing meaningful, quantitatively expressed (or
ACaseofAdoptionof25000StandardsFamily-EstablishingEvaluationRequirementsintheAudio-VisualPreservation
Context
225
measurable) indicators associated to quality
characteristics (Software Engineering Institute,
2006).
These two standards (ISO/IEC9126 and ISO/IEC
14598) are converged into the ISO/IEC 25000,
SQuaRE - Software Product Quality Requirements
and Evaluation - series (ISO/IEC FDIS 25000,
2005). It represents families of standards and
constitutes the result of an effort to harmonize
previous standards in order to establish criteria for
the specification of software products quality
requirements, their measurement and evaluation.
The general objective for SQuaRE is to respond
to the evolving needs of users (those who develop
and those who acquire software products) through an
improved and unified set of normative documents
covering three different and complementary quality
processes: requirements specification, measurement
and evaluation (Suryn Witold, Abran Alain, 2003).
SQuaRE consists of five thematic areas called
divisions: Quality Management, that contains all
common models, terms and definitions referred to
by all other standards from the SQuaRE series,
Quality Model, that presents detailed quality
models including characteristics for internal,
external and quality in use, Quality Measurement,
that includes a software product quality
measurement reference model, mathematical
definitions of quality measures, and practical
guidance for their application, Quality
Requirements, that helps to specify quality
requirements, based on quality models and quality
measures, and Quality Evaluation, that provides
requirements, recommendations and guidelines for
software product evaluation, whether performed by
evaluators, acquirers or developers.
Other standards involved in the quality
assessment are: ISO/IEC 15939 standard “Systems
and software engineering - Measurement process”
(ISO/IEC 15939, 2007), that defines a measurement
process applicable to system and software
engineering and management disciplines, the
ISO/IEC/IEEE 12207 “Systems and software
engineering — Software life cycle processes”
(ISO/IEC/IEEE 12207, IEEE, 2008) which focuses
on the processes in the life cycle of a software
product or service and the ISO/IEC/IEEE 15288
“Systems and software engineering - System life
cycle processes” (ISO/IEC/IEEE 15288, IEEE,
2008), which targets the processes in the life cycle
of a system. Finally there is ISO/IEC 15504,
“Information technology — Process assessment”
(ISO/IEC 15504, 2004) (noted as SPICE - Software
Process Improvement and Capability dEtermination)
which deals with process assessment and its
implementations in terms of guidelines and tools.
3.2 IEEE Standards
The IEEE standards encompass software and
systems lifecycles, from concept and development to
delivery and maintenance. Also the reuse of
software components is included. The most
important are: the standard IEEE 830
“Recommended Practice for Software Requirements
Specifications” (IEEE 830, 1998) which focus on
content and characteristics of a Software
Requirements Specification, the standard IEEE 1012
“Software Verification and Validation” (IEEE 1012,
2004) which defines the Verification & Validation
(V&V) processes in terms of specific activities and
includes the (ISO/IEC/IEEE 12207, IEEE, 2008)
and the standard IEEE 1074 “Standard for
Developing Software Life Cycle Processes” (IEEE
1074, 2006) which defines the process activities
mandatory for the development and maintenance of
software.
4 QUALITY EVALUATION
PROCEDURE
In this section the experience matured inside the
Presto4U project for the definition of the software
quality evaluation procedure is reported. This
activity should ideally involve multiple stakeholders
and provide information to each of them to use.
Inside Presto4U, for example, in tune with its
nature and particular interests, each community of
practices worked for producing a customized set of
preservation needs to be given to software quality
expert evaluators. According to a bottom-up
approach, on the basis of detected preservation
needs, the evaluators have defined the preservation
requirements specified using a quality model as
reference and have mapped the requirements on
characteristics and sub-characteristics of this quality
model. In this paper, the exploratory case study
stops at this phase, producing an intermediate result
that is a high-level software product quality
evaluation plan.
Subsequently, the plan will be refined with
measures, techniques, criteria, etc. On the basis of
the implementation degree of the detected in the
plan characteristics and sub-characteristics, the used
by each CoP tools will be evaluated.
ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications
226
In spite of the results will be preliminary, the
most important for each CoP quality aspects will
been highlighted and defined.
After all, the quality is highly context-dependent,
because it means different things to different people
(Kitchenham, Pfleeger, 1996).
The aspects considered during the requirements
elicitation activity have been:
the definition of the project scope: digital
audio-visual preservation;
the definition of the project purpose: the project
aims to develop a body of knowledge on the
status of digital preservation practice
(problems and needs), to map preservation
needs in quality characteristics, to evaluate the
used tools, to identify useful research results
and to promote their adoption and
implementation;
the definition of the different products that
could be considered as a target for the quality
evaluation procedure: in the audiovisual
preservation context, they can be either tools
for mapping between metadata formats and
standards or for archiving and restoring of
audiovisual files, or for automatic extracting
and enriching of metadata of audiovisual
contents, or for evaluating the quality of the
various contents (like images and sounds).
On the basis of these elements and the
availability of the involved stakeholders, the first
steps of a quality evaluation process have been
performed and the most important quality aspects in
the digital audio-visual preservation context have
been highlighted.
4.1 Establishing the Evaluation
Requirements
Inspired by the ISO/IEC 25040 (ISO/IEC 25040,
2010) and by the software product quality evaluation
process that it contains, in this section the activity of
“establishing the evaluation” requirements adopted
inside the Presto4U is presented. In the project, as
target of preservation products, software tools have
been considered.
Without the pretense to define general results,
here below the identified steps useful for producing
a high level software product quality evaluation plan
are presented. Inside a complete software product
quality evaluation process, this type of plan
constitutes an outcome of the first phase of the
process, called “establish the evaluation
requirements”, and an input of the second phase
called “specify the evaluation”. Running parallel to
the project development, currently this work is
placed in the first phase of the evaluation process,
and the high level software product quality
evaluation plan is just an important intermediate
result.
For the phase called “establish the evaluation
requirements”, different steps have been identified
and every step has been implemented as follows:
1. Establishing the purpose of the evaluation:
As a purpose of the project was evaluating the
used tools in order to identify useful research
that could promote their adoption, purpose of
the evaluation is reconsidering the
acceptability of these products.
2. Obtaining the software product quality
requirements:
In order to ensure that the products meet user
and customer needs, stakeholders were
detected for collecting information and a
quality model was defined as reference
because software product quality requirements
shall be specified using a quality model.
About the stakeholders, in the evaluation
process, they were identified among the staff
of the CoPs. By interviews and
questionnaires, a list of audio-visual
preservation needs came to light. In particular,
for overcoming initial differences between
basic terminologies (software engineering vs
audio-video preservation), a shared and
informal glossary has been preliminary
defined preliminary. For confidential reasons
this glossary can not be included in this paper.
On the basis of detected preservation needs, a
list of audio-visual preservation requirements
has been defined using as quality model the
product quality model reported in standard
ISO/IEC 25010 and below shown.
3. Identify product parts to be included in the
evaluation:
Inside the audio-visual preservation
environment different products could be
considered as a target for the quality
evaluation. They can be either tools for
mapping between metadata formats and
standards or for archiving and restoring of
audiovisual files, or for automatic extracting
and enriching of metadata of audiovisual
contents, or for evaluating the quality of the
various contents (like images and sounds).
Besides, each product can have specific
preservation quality requirements due to the
target usage or its different nature (product
purchased, in a development stage, fully
ACaseofAdoptionof25000StandardsFamily-EstablishingEvaluationRequirementsintheAudio-VisualPreservation
Context
227
developed, etc.) or the hardware, software and
network environment in which the product
will be used. Overall, these products cover
different tasks that have to be addressed
throughout the preservation workflow of
digital audiovisual media. The types of
identified products are tools that perform
different quality aspects. For example, in the
audio-visual preservation context, these
quality aspects could refer to the quality of the
content (“are the images good and the sound
clean?”) but also to the validity of the files
(“are they adherent to the file standards and
correctly playable?”, “Content can be
accessed on master archival items or on items
produced for dissemination?”). The parts of
tools that could response to these questions
are these parts to be included in the
evaluation.
4. Define the stringency of the evaluation in order
to provide confidence to it:
As the evaluation stringency should be related
to a set of characteristics and sub-
characteristics that establish the expected
evaluation levels, the detected quality
requirements have been mapped on the
characteristics and sub-characteristics of the
detected quality model, which is the product
quality model of ISO/IEC 25010 (ISO/IEC
FDIS 25010, 2011).
The mapping of the preservation requirements
into the quality model constitutes a preliminary high
level software product quality evaluation plan.
Subsequently to quality evaluation plan
development, in the phase called “specify the
evaluation”, the evaluation measures to be applied,
the decision criteria to be defined and the evaluation
results to be achieved will be detailed. However, this
phase and those that follow (ISO/IEC 25040, 2010)
are out of the scope of this paper.
Below, the product quality model is shown.
4.2 Quality Model
The standard ISO/IEC 25010 proposes two quality
models for software product evaluation: quality in
use model and product quality model. Inside the
25000 standard family, measures for each quality
model are proposed (ISO/IEC 25022, 2012, ISO/IEC
25023, 2012). The product quality was adopted as
quality model reference and in table 1 it is shown.
The model consists of eight characteristics and each
characteristic is composed of a set of related sub-
characteristics. These characteristics and sub-
characteristics will be the basis of the definition of
the specialized high level quality plan according to
needs and requirements of the CoPs. Therefore, the
specialization could include the reduction of the sets
of characteristics and sub-characteristics depending
on the software tools to be evaluated and the CoPs
constrains. An example of this specialization is
presented in Section 5.
Table 1:Product Quality Model.
Product Quality Model
Characteristics Sub-Characteristics
Functional suitability
Functional completeness
Functional correctness
Functional appropriateness
Performance efficiency
Time behavior
Resource utilization
Capacity
Compatibility
Co-existence
Interoperability
Usability
Appropriateness
recognisability
Learnability
Operability
User error protection
User interface aesthetics
Accessibility
Reliability
Maturity
Availability
Fault tolerance
Recoverability
Security
Confidentiality
Integrity
Non-repudiation
Accountability
Authenticity
Maintainability
Modularity
Reusability
Analyzability
Modifiability
Testability
Portability
Adaptability
Installability
Replaceability
5 EXPLORATIVE CASE STUDY
In this section, the steps of the activity presented in
Section 4 are exploited in order to develop the
quality evaluation plan for the TATE Gallery as
partner representative of the Video Art, Art Museums
and Galleries CoP (Presto4U). Therefore, in the case
study, the considered stakeholders have been
identified among the staff of TATE Gallery.
ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications
228
As reported in the section 4 about quality process
evaluation, the “establishing the evaluation
requirements” is a preliminary activity of the
evaluation process.
Possible inputs for this activity should be:
Software product quality evaluation needs
Software product quality requirements
specification
Applicable measurement tools and
methodology
Software product to be evaluated including
intermediate products
Possible outcomes for this activity should be:
Specification of software product quality
evaluation purposes
Specification of software product quality
evaluation requirements
Specification of high level software product
quality evaluation plan
To derive the quality plan the following steps
have been defined:
1. Definition of preservation needs: by interviews
and questionnaires, a list of preservation needs
should be generated.
2. Identification of preservation requirements:
through an iterative refinement process the
preservation requirements should be identified
from the declared needs.
3. Definition of the preservation quality plan: the
preservation requirements should be mapped
in some characteristics and sub-characteristics
of the product quality model (ISO/IEC FDIS
25010, 2011). As result the high level
software product quality evaluation plan is
obtained.
In the rest of this section further details of the
implementation of these three steps in the considered
case study are provided.
5.1 Preservation Needs
As the case study presented in this paper has been
developed considering the experience and the
knowledge of the TATE Gallery partners, the
identified preservation needs are therefore specific
of useful for the digital video art collections
software tools. Obviously, the list is partial and
further implementable. In the following the first
draft of the list of collected needs is reported:
1. The software should perform the functions of
data ingest, archival storage and migrating
digital file to new formats or carriers when
necessary.
2. The software should be modified without any
impact on existing quality
3. The software should be interoperable with
TATE Gallery collection management system
4. All copies must be recorded as components on
the collection management system (TMS) and
the purpose and status of a particular copy
must also be recorded
5. Any access to the file is restricted
6. Location information must be accurate and
kept up to date and record the presence of a
digital file.
7. Ingest and storage activities must not
compromise the video quality of the video
material. The original video quality must be
preserved.
8. Preservation actions and decisions must be
documented and transparent.
9. Every action or modification on the media files
must be recorded and traced
10. Looking, controlling and operating on the
system content
11. The software should be able to be transferred
in different broadcast environments and to be
adaptable to their specific exigencies
The list has been then elaborated to derive the set
of preservation requirements as detailed in the next
section.
5.2 Preservation Requirements
The second step of the proposed methodology has
been the identification of the preservation
requirements. Usually the requirements can be
written either in natural language or in semi-formal
language (with graphical notations, precise syntax
and a non-rigorous semantic) or in formal language
(mathematics-based language with syntax and
semantics formally defined). Among them, the first
is the most common and easy way to express
software requirements despite the ambiguity risk
(Berry, D.M., Kamsties, E., Krieger, 2003).
Expressed in natural language requirements can be
easy communicated and discussed among various
technical and no-technical stakeholders, before
being used in the subsequent product development
phases. In this case study, natural language
requirements have been necessary for the
discussions among project partners that operate in
different from software engineering fields.
From the previous list of preservation needs, a
list of requirements has been refined.
ACaseofAdoptionof25000StandardsFamily-EstablishingEvaluationRequirementsintheAudio-VisualPreservation
Context
229
Req. 1: The software has to perform all the
functionalities that it is developed for.
Req. 2: The software should let the possibility to
integrate/correct/modify user-specific features or
components without any impact on existing product
quality.
Req. 3: The tool should be interoperable with
different collection management systems and should
let the possibility to import data from different
format.
Req. 3.1: The tool should import material in both
25Hz and 29.97Hz formats.
Req. 3.2: The tool should input/output different
file formats like for instance MXF.
Req. 4: The software should track all copies and
record the purpose or status of each copy.
Req. 5: Any access to the file is restricted.
Req. 6: Location information must be accurate
and traceable.
Req 6.1: The tool should store different
information about the file considered such as: Object
No, Title, Artist, Dims, Comp No, Comp Name,
Comp Desc, Media Fmt, Duration, Video Std,
Aspect Ratio, When Made?, Where made?,
Provenance, TiBM label, Current Location, Video
Res., Colour, Audio, Details of Master, Misc,
Author.
Req. 7: During its activity, the software should
not allow the alteration of the ingested material
video quality.
Req. 7.1: The system should include a quality
control flags, monitor the level of errors, and support
corrections when occurred.
Req. 8: Preservation actions and decisions can be
proven to have taken place and cannot be repudiated
later.
Req. 9: Every action or modification on the
media files must be recorded and traced.
Req.10: The software should let easy control and
operate on the content.
Req.11: The software should be adapted for
different, evolving, operational or usage
environments.
5.3 Towards an Evaluation Plan:
Mapping between Requirements
and Product Quality Model
The last step of the proposed methodology is the
definition of a preservation quality evaluation plan.
Considering the main just above-cited requirements
and the characteristics and sub-characteristics of the
adopted quality model (see paragraph 4.2), here
below a possible mapping between these elements is
reported.
For completeness, requirements and definitions
of the sub-characteristics are reported.
Req. 1 – The software has to perform all the
functionalities that it is developed for.
Characteristic: functional suitability
Sub-characteristic: functional completeness -
Degree to which the set of functions covers all the
specified tasks and user objectives.
Req. 2 – The software should let the possibility to
integrate/correct/modify user-specific features or
components without any impact on existing product
quality.
Characteristic: maintainability
Sub-characteristic: modifiability - Degree to
which a product or system can be effectively and
efficiently modified without introducing defects or
degrading existing product quality.
Req. 3 – The tool should be interoperable with
different collection management systems and should
let the possibility to import data from different
format.
Characteristic: compatibility
Sub-characteristic: interoperability - degree to
which two or more systems, products or components
can exchange information and use the information
that has been exchanged.
Req. 4 – The software should track all copies
and record the purpose or status of each copy.
Characteristic: reliability
Sub-characteristic: availability - degree to which
a system, product or component is operational and
accessible when required for use.
Req. 5 – Any access to the file is restricted.
Characteristic: security
Sub-characteristic: confidentiality - degree to
which a product or system ensures that data are
accessible only to those authorized to have access.
Req. 6 – Location information must be accurate
and traceable.
Characteristic: security
Sub-characteristic: authenticity - degree to which
the identity of a subject or resource can be proved to
be the one claimed.
Req. 7 – During its activity, the software should
not allow the alteration of the ingested material
video quality.
Characteristic: usability
Sub-characteristic: user error protection – degree
to which a system protects users against making
errors.
ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications
230
Req. 8 – Preservation actions and decisions can
be proven to have taken place and cannot be
repudiated later.
Characteristic: security
Sub-characteristic: non-repudiation - degree to
which actions or events can be proven to have taken
place, so that the events or actions cannot be
repudiated later.
Req. 9 – Every action or modification on the
media files must be recorded and traced.
Characteristic: security
Sub-characteristic: accountability - degree to
which the actions of an entity can be traced uniquely
to the entity.
Req.10 – The software should let easy control
and operate on the content.
Characteristic: usability
Sub-characteristic: operability - degree to which
a product or system has attributes that make it easy
to operate and control.
Req.11 – The software should be adapted for
different, evolving, operational or usage
environments.
Characteristic: portability
Table 2: High level software product quality evaluation
plan.
Product Quality Model
Requirements
Characteristics
Sub-
characteristics
Functional
suitability
Functional
completeness
REQ.1
Compatibility Interoperability REQ.3
Usability
Operability REQ. 10
User error
protection
REQ. 7
Reliability Availability REQ.4
Security
Confidentiality REQ. 5
Non-repudiation REQ. 8
Accountability REQ. 9
Authenticity REQ.6
Maintainability Modifiability REQ.2
Portability Adaptability REQ.11
Sub-characteristic: adaptability - degree to which
a product or system can effectively and efficiently
be adapted for different or evolving hardware,
software or other operational or usage environments.
The below-presented table summarizes a
specialized high level quality evaluation plan that
will have to be refined with techniques, measures to
be applied, decision criteria to be defined and
evaluation results to be achieved for better
specifying the evaluation. In spite of its high level
detail, it is just an example of quality evaluation
process refinement on only few of the possible
characteristics and sub-characteristics of the quality
model. The refinement highlights the most important
quality aspects in the specific context of digital
video art collections, which are mainly related to
security.
6 DISCUSSION AND
CONCLUSION
The paper has presented an experience in the
identification of audio-visual preservation needs and
corresponding requirements in order to set up a
quality evaluation process for tools and applications
identified inside the Presto4U project.
Considering in particular the digital collections,
the definition of specific preservation needs has been
performed in collaboration with TATE Gallery.
Subsequently the specification of these needs in
software product quality evaluation requirements
and the identification of a quality model for mapping
them have been implemented. For this a selection of
the most suitable characteristics and sub-
characteristics of the detected quality model came to
light and a preliminary high level software product
quality evaluation plan has been defined.
The experience reported in this paper confirms
some of the weaknesses identified by different
researchers in the procedure of international
standards applicability. In particular most problems
rise in the alignment of the classical terminology of
the software engineering context with the one that is
specific for the preservation environment. Thus a
shared and informal glossary has been preliminary
defined so to avoiding possible misunderstandings.
Moreover, this experience highlights the necessity of
a quality evaluation plan customized according to
specific needs to be considered and specific
characteristics to be measured. Indeed the list of
characteristics and sub-characteristics provided in
the standards do not completely reflect the specific
exigencies of the particular environments as, for
example, the preservation environment. In line with
ACaseofAdoptionof25000StandardsFamily-EstablishingEvaluationRequirementsintheAudio-VisualPreservation
Context
231
one of the research results about the applicability of
the standards, it is opportune underlining the
necessity of specializing quality evaluation plan by
selecting the characteristics to be measured taking
also in consideration the organizational limits and
constraints.
The procedure customized for the audio-visual
preservation has been positively accepted inside the
Presto4U project that, for the first time, faces the
problems of the applicability of international
standards for software quality assessment. Different
stakeholders have considered the proposed
procedure, the encountered problems and the
practical proposed solutions a good reference to
replicate the experience in software engineering
contexts different from audio-visual preservation
one.
Besides, the proposed quality evaluation plan is
part of the future work of assessment of the audio-
visual preservation tools. Similarly to what has been
done for the definition of the quality evaluation plan,
the set of measures will be also specialized and
adapted to the audio-visual preservation context.
For this a wide range of different quality aspects
would be taken into account such as: properties of
source code, test-related properties such as coverage,
or architecture-related properties, such as data access
constraints or communication directives. The
specific information needs of the different involved
in Software Quality Assessment (SQA) holders are
another essential dimension.
ACKNOWLEDGEMENTS
The authors would like to thanks Pip Laurenson of
TATE Gallery for the useful contributions and
discussions.
This work has been partially funded by the
European Technology for Digital Audiovisual Media
Preservation (Presto4U) FP7 project contract
n.600845.
REFERENCES
Abran A., R. E. Al-Qutaish, and J.-M. Desharnais, 2005,
Harmonization issues in the updating of the ISO
standards on software product quality, Metrics News:
Journal of the Software Metrics Community, Vol. 10,
No. 2, pp. 35- 44.
Abran A., R. E. Al-Qutaish, J.-M. Desharnais, and N.
Habra, 2005, An information model for software
quality measurement with ISO standards, Proc. of the
International Conference on Software Development
(SWDC-REK'05), Reykjavik, Iceland, pp. 104-116.
Abran A., R. E. Al-Qutaish, and J. Cuadrado-Gallego,
2006, Analysis of the ISO 9126 on Software Product
Quality Evaluation from the Metrology and ISO 15939
Perspectives, WSEAS Transactions on Computers,
Vol. 5, No. 11, pp. 2778-2786.
Azuma Motoei. 2011. The impact of ICT evolution and
application explosion on software quality: a solution
by ISO/IEC 250nn square series of standards. In Proc.
of the 8th international workshop on Software quality
(WoSQ '11). ACM, New York, NY, USA, 1-2.
Berry, D.M., Kamsties, E., Krieger, M.M., 2003: From
contract drafting to software specification: Linguistic
sources of ambiguity. TR, University of Waterloo.
Bhatti A. M., H.M. Abdullah, and C. Gencel, 2009, A
Model for Selecting an Optimum Set of Measures in
Software Organizations, Proc. of EuroSPI 2009, CCIS
42, pp. 44–56, 2009., Berlin Heidelberg.
Bundschuh, M., Dekkers, C., 2008: The Measurement
Compendium: Estimating and Benchmarking Success
with Functional Size Measurement. Springer,
Heidelberg.
Chua B. B. and L. E. Dyson, 2004, Applying the ISO 9126
Model to the Evaluation of an e-Learning System,
Proc. of the 21st Conference of the Australasian
Society for Computers in Learning in Tertiary
Education (ASCILITE'04), Perth, Australia, pp. 184-
190.
Gopal, A., Krishnan, M.S., Mukhopadhyay, T.,
Goldenson, D.R., 2002: Measurement Programs in
Software Development: Determinants of Success.
IEEE Transactions on Software Engineering 28(9),
863–875.
Hall, T., Fenton, N., 1997: Implementing Effective
Software Metrics Programs. IEEE Software 14(2),
55–65.
IEEE 830, 1998, Recommended Practice for Software
Requirements Specifications
IEEE 1012, 2004, System and Software Verification and
Validation
IEEE 1074, 2006, IEEE Standard for Developing Software
Life Cycle Processes
ISO 9000-3,1991, Quality management and quality
assurance standards -- Part 3: Guidelines for the
application of ISO 9001 to the development, supply
and maintenance of software
ISO 9126, 1991, Information Technology, Software
Product Evaluation, Quality characteristics and
guidelines for their use.
ISO 14721, 2012, Space data and information transfer
systems -- Open archival information system (OAIS) --
Reference model.
ISO/IEC 9126 – 1, 2001, Software engineering - Product
Quality Part 1: Quality Model.
ISO/IEC 9126 – 2, 2001, Software engineering - Product
Quality Part 2: External Metrics.
ISO/IEC 9126 – 3, 2001, Software engineering - Product
Quality Part 3: Internal Metrics.
ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications
232
ISO/IEC 9126 – 4, 2001, Software engineering - Product
Quality Part 4: Quality in use metrics.
ISO/IEC 14598 (parts 1 to 6), 1998, Software engineering
— Software Product evaluation
ISO/IEC 15504, 2004, Information technology — Process
assessment
ISO/IEC 15939, 2007, Systems and software engineering -
- Measurement process.
ISO/IEC FDIS 25000, 2005, Systems and software
engineering — Systems and software Quality
Requirements and Evaluation (SQuaRE)
ISO/IEC FDIS 25010, 2011, Systems and software
engineering — (SQuaRE) — System and software
quality models.
ISO/IEC 25022, 2012, Systems and software engineering -
SQuaRE – Measurement of quality in use.
ISO/IEC 25023, 2012, Systems and software engineering -
SQuaRE – Measurement of system and software
product quality.
ISO/IEC 25040, 2010, Systems and software engineering
- SQuaRE – Evaluation process.
ISO/IEC/IEEE 12207, IEEE, 2008, Systems and software
engineering — Software life cycle processes.
ISO/IEC/IEEE 15288, IEEE, 2008, Systems and software
engineering — System life cycle processes.
Kitchenham B, Pfleeger S. L., 1996, Software quality: the
elusive target, IEEE SOFTWARE 0740-7459/96,
1996 IEEE Vol. 13, No. 1: January, pp. 12-21.
Koscianski A. and J. C. Bracarense Costa, 1999,
Combining Analytical Hierarchical Analysis with
ISO/IEC 9126 for a Complete Quality Evaluation
Framework, Proc. of the 4th IEEE International
Symposium and Forum on Software Engineering
Standards (ISESS'99), Curitiba, Brazil, pp. 218-216.
Krause P., Freimut B., Suryn W., 2003, New Directions in
Measurement for Software Quality Control, Proc. of
STEP2002, Computer Society Press.
Olsina L., Philip Lew, Alexander Dieser, and Belen
Rivera. 2012. Updating quality models for evaluating
new generation web applications. Journal Web Eng.
11, 3 September, 209-246.
Pfleeger S.L., 2009, Software Engineering, Theory and
Practice, Fourth Edition, Prentice Hall.
Polillo R.. 2012. A core quality model for web
applications. Journal Web Eng. 11, 3 September, 181-
208.
Presto4U project: www.prestocentre.org/4u
Rafa E. Al-Quataish, 2009, An Investigation of the
Weaknesses of the ISO 9126 International Standard,
Proc. of 2nd International Conference on Computer
and Electrical Engineering, IEEE, pp.275 – 279
Software Engineering Institute, 2006, The International
Process Research Consortium: A Process Research
Framework, December, 20-28.
Suryn Witold, Abran Alain, 2003, ISO/IEC SQuaRE. The
second generation of standards for software product
quality, Proc. of IASTED 2003 – SEA 2003, November
3-5, Marina del Rey, CA, USA.
TATE Gallery: www.tate.org.uk/
Vliet, H., 2002, Software Engineering, Principles and
Practice, Second Edition. John Wiley & Sons.
Zádor Dániel Kelemen, Katalin Balla, Jos Trienekens, Rob
Kusters, 2008 Towards supporting simultaneous use of
process-based quality approaches, Proc. of 9th
International Carpathian Control Conference pp. 291-
295 Sinaia Romania.
ACaseofAdoptionof25000StandardsFamily-EstablishingEvaluationRequirementsintheAudio-VisualPreservation
Context
233
