VALUATION FRAMING FOR SOCIAL SOFTWARE
A Culturally Aware Artifact
Roberto Pereira and M. Cecilia C. Baranauskas
Institute of Computing, University of Campinas, Av. Albert Einstein N1251, Campinas-SP, Brazil
Keywords: Organizational Semiotics, Culture, Values, Social Software.
Abstract: Despite the popularity of the so-called social software, just a small fraction of the systems launched on the
Web is really successful. The diversity of users, their limitations, preferences, values and culture, are
examples that indicate the complexity of developing this kind of system; moreover there is still a lack of
approaches, artifacts and methods for supporting designers to deal with this complexity. This paper presents
an artifact specially adapted to support designers in the task of evaluating social software, taking values and
cultural issues into account. It draws on Organizational Semiotics and on building blocks of culture to shed
light on this research area. The artifact was applied to the evaluation of five different prototypes of systems
for supporting cross-cultural collaboration, and the results demonstrate the viability of using this artifact for
supporting the evaluation as well as the design of social software.
1 INTRODUCTION
Social software can be understood as systems that
allow people, in their particularities and differences,
to communicate (interact, collaborate, share ideas
and information), mediating and facilitating any kind
of social relationship; systems whose usefulness is
dependent on and whose structure is shaped by the
active participation, interaction and production of
content by their users (Pereira et al., 2010).
The term social software is usually used in many
different contexts, and different technologies are
covered by it. As Lazar and Preece (2003) claim in
the context of Online Communities, we can say that
social software is usually a subjective matter, easy to
understand and recognize, but unstable to define and
measure and even more complicated to evaluate.
After the Web 2.0 advent, new applications
allowing mass collaboration, communication and
interactivity, such as YouTube, Delicious, Twitter,
Flickr, Facebook among others, were developed.
These systems, named social software, invite
millions of users to communicate, interact, create,
share and organize information. They show the
“power of the collective”, the opportunities and
knowledge that can be generated through
collaborative work and through mass interaction.
Social software were considered a mark of a web
paradigm-shift, where more than connecting pages
and resources the web became a connection of
people and organizations — a social web.
In the previously cited systems, the interaction
occurs in an unprecedented scale and intensity,
leading to a situation in which issues related to
human-computer interaction (HCI) are extended to
issues related to human-computer-human interaction
in social situations. Actually, social software made it
visible part of the transformations that have
redefined people’s relationship with technology. As
Sellen et al. (2009) point out, people now live with
technology, not just use it; they are increasingly
hyperconnected, increasingly dependent on
technology and the information produced by them is
becoming less ephemeral.
In this sense, as technology left the context of
offices and workplaces to pervade every aspect of
people’s personal and social lives, a broad set of
factors that range from emotional and affective
aspects, sociability and human values, to issues of
scalability, security and performance are now in
play. This new and complex scenario brings
challenges that research communities and
practitioners, in not only HCI, Collaborative
Systems and Software Engineering, but also in
Databases, Computer Networks and other areas
related to technical infrastructure, have never faced
before. Moreover, these challenges are reflected in
the emergent interest and need for involving other
235
Pereira R. and Cecilia C. Baranauskas M..
VALUATION FRAMING FOR SOCIAL SOFTWARE - A Culturally Aware Artifact.
DOI: 10.5220/0003479002350244
In Proceedings of the 13th International Conference on Enterprise Information Systems (ICEIS-2011), pages 235-244
ISBN: 978-989-8425-56-0
Copyright
c
 2011 SCITEPRESS (Science and Technology Publications, Lda.)

fields that go beyond computing, such as sociology,
psychology, anthropology, communication, etc.
Indeed, despite the popularity and growing in the
number of users of social software, just a small
fraction of these systems is really successful. Being
completely dependent on their users, the success of
social software heavily depends on how users feel
when using them, on their interface features and on
their interaction mechanisms. As Neris et al. (2009)
suggest, for developing systems that fully meet
users’ requirements, we need to know users in their
abilities and culture, formalizing the interaction
requirements and investigating solutions of
interaction/interface for the diversity. In fact,
systems should reflect an understanding (and
respect) about people’s values, preferences,
limitations and behaviors, including the way people
actually interact, play, learn, work, and live in their
organizations, groups, communities and other forms
of societal life. Otherwise, as Ackerman (2000)
asserts, the produced systems will be useless,
inefficiently automating the collaboration,
communication and other social activities.
Although the social software context is clearly
recognized as complex and challenging, research
initiatives on guidelines, methods, tools and even
theories for supporting designers are still incipient.
According to Hendler et al. (2008), a web
application should be understood as a “social
machine” which includes an underlying technology,
but also rules, strategies and organizational
structures used to manage the technology. This
vision requires investigation in social software from
two perspectives: as a social phenomenon in a macro
level and as a technological artifact to be built in a
micro one. As a consequence of these perspectives,
the software development life cycle, which has been
traditionally based on best practices in Software
Engineering (specification, design, construction,
testing, etc.), needs to be rethought. Cultural issues
must be considered in an explicit and transverse
way; the process has to be aware of the values of
people who will be direct or indirectly affected by
the development, deployment and use of the system.
Similarly, traditional concepts and practices in HCI,
such as usability and accessibility, need to be put
into perspective and understood as technical values
crucial to the project of any technological artifact.
Values are desirable, trans-situational goals,
varying in importance and serving as guiding
principles in people’s lives (Schwartz, 2005). Hall
(1959) explains that every innovation, e.g. a social
software, brings negative and positive impact to the
environment in which it is introduced. Indeed,
because people’s values are culturally built, we
argue that people’s culture influences the way an
innovation will be valued by its direct and indirect
users, being determinant in the appropriation or
rejection of that innovation.
In this paper, we highlight the importance of
taking people’s culture and values into account when
designing and evaluating social software and present
a culturally aware artifact for analyzing them: the
Valuation Framing (VF) (Kolkman 1993). This
artifact, from the Organizational Semiotics Theory
(Liu, 2000), was specially modified for the context
of social software by explicitly suggesting values
related to the context of this kind of system — we
are naming it VF4SS.
The paper is organized as follows: section 2
presents a brief literature review on social software;
section 3 presents the VF4SS as an artifact for
analysing social software, taking into account
people’s culture and making values an explicit issue;
section 4 describes our findings when using it for the
evaluation of five different projects during their
design phase; section 5 presents our conclusions and
directions for future research.
2 LITERATURE REVIEW
When we talk about social software we are not just
talking about a specific set of technologies for which
the main focus is on people. Rather, as Boyd (2007)
points out, we are talking about a movement in
which there are three significant changes: the first is
the way technology is developed — the perpetual
beta instead of locked-down versions; the second is
the way participation is widespread — the network
effect and organic growth; and the third is the way
people behave — the focus is on connecting people
and watching the subject and shared interests
emerging through that instead of creating pre-
defined groups.
For Webb (2004), the main particularity of social
software is in the design process because human
factors and group dynamics introduce design
difficulties that are not obvious without considering
the human psychology and nature. The success and
usefulness of social software rely directly on their
users and, therefore, on aspects related to the user
experience, such as emotional and socio-technical
factors, including how the interface was designed.
Therefore, it is urgent to discuss these concepts
considering human values of a mutable society
where users are not only consumers, but also
creators of content and programmers of mashup;
ICEIS 2011 - 13th International Conference on Enterprise Information Systems
236

where the technology should allow a creative
involvement and consider the emotional aspects of
the user experience; and where the interaction via
Web can happen anytime, anywhere and from
computer systems embedded in different objects.
However, neither the traditional approaches for
software development nor the methods and tools for
supporting software evaluation and analysis are able
to deal with social software in its complex scenario.
According to Thompson and Kemp (2009),
traditional methods for usability evaluation, such as
Heuristics Inspection, do not consider key-aspects of
social software, such as technological aspects (e.g.,
scalability, collaboration) and those related to the
users’ experience (e.g., the quality of the produced
contents and the interactions among users). The
authors are based on previous studies by other
researchers and conduct experiments to identify
aspects that, although fundamental to determine the
success of an application, are often not considered.
As an effort in understanding the social software
nature, Smith (2007) proposes a functional
framework composed of elements (e.g., identity,
groups, reputation) that have been identified by
researchers and professionals interested in the design
and evaluation of social software. According to
those authors, social software have a set of common
elements that are combined and implemented in
order to produce different environments. Although a
good starting point, the framework was limited to
functional aspects ignoring those related to
sociability, values and other cultural issues. For
instance, concepts such as accessibility, autonomy
and collaboration could not be forgotten or neglected
in a social software design and evaluation, but the
framework does not draw attention to them.
In the context of social software design, we
developed a discussion regarding the elements that
compose social software, approaching them in terms
of informal (e.g., personal), formal (e.g., social or
collective) and technical values (Pereira et al.,
2010), and presented a set of values identified
through technical analysis and an extensive literature
review. This set encompasses technical as well as
ethical, personal and collective aspects, and draws
attention to their differences and interactive nature.
The main idea is that depending on which values are
prioritized, how these values are combined and how
they are technically supported, quite different
environments which promote certain values while
inhibit others will be produced.
As Friedman (1996) asserts, the cost to
disseminate a technology is insignificant when
compared to the cost to develop it; moreover the
values embedded in its implementations are deep,
systematic and easily disseminated. To her, although
the neglect of moral values in any organization is
disturbing, it is particularly damaging in the design
of computer technology, because, unlike people with
whom users can disagree and negotiate about values
and their meanings, they hardly can do the same
with technology. In this sense, the set of values
suggested in (Pereira et al., 2010) can support
designers, evaluators and analysts to keep values in
mind mainly when the project of a social software is
in its early phases; when they need to evaluate a
social software and do not have any guide; or even
when there is no time or resources for carrying out a
deep analysis regarding the values involved.
Regarding values in technology design,
Friedman et al. (2006) present the Value Sensitive
Design (VSD): a methodology for involving human
values in the project of technologies. Although
pioneer in bringing the subject of values to scene,
this methodology is concerned mainly with values of
moral nature and still needs artifacts and tools for
supporting designers to use it in a practical context.
In fact, Harrison et al. (2007) and Sellen et al.
(2009) highlight the need for developing and
publishing studies in order to support designers and
evaluators to deal with the complexity and different
requirements that current technologies have. In
agreement to them, Miller et al. (2007) state that if
designers and developers in fast-paced and bottom-
line oriented industry settings are to account for
values, they must be provided with light-weight and
principled methods to do so.
Adopting this view and arguments, we classified
the values identified in the context of social software
(Pereira et al., 2010) according to their cultural
nature and incorporated them into the VF artifact
(Kolkman 1993) creating the VF4SS. The next
section presents both artifacts and shows how they
can be used for evaluating social software through
the lenses of values and cultural aspects.
3 THEORETICAL
AND METHODOLOGICAL
BASES
According to Hall (1959), humans operate at three
different levels: informal, formal and technical. Each
level is present in any situation, but one will always
dominate in a given instant of time. Sometimes, the
shifts (and boundaries) between these levels are
subtle and rapid, but understanding these shifts is the
VALUATION FRAMING FOR SOCIAL SOFTWARE - A Culturally Aware Artifact
237

basic requirement to understand the process of
change.
In the Organizational Semiotics (OS) theory
(Liu, 2000), an organization and its information
system are considered a social system in which
human behaviours are organized by a system of
norms. In this theory, any technological artifact
(e.g., a social software) is embedded in a formal
system which, in turn, exists in the context of an
informal one. The Semiotics Onion is an artifact of
the OS that represents Hall’s (1959) three levels (see
Figure 1): the informal, where the organizational
culture, customs and values are reflected as beliefs,
habits and individual behaviour patterns of its
members; the formal in which rules and procedures
are created to replace meanings and intentions; and
the technical that represents the computer system
situated within the formal level.
Figure 1: The semiotics onion.
Traditionally, the design process of technological
artifacts occurs regardless the formal and informal
aspects of organizations and the society. That is,
technological innovations are produced and
delivered for people to use them even without a clear
perception of their utility and potential impact: it
starts and finishes in the core of the Semiotics
Onion. Grounded on OS theory, Baranauskas (2009)
claims that we need discard this limited view in
favour of one that understands the design process
from a social perspective (see Figure 1): “as a
movement that starts in the society, crosses the
informal and formal layers of signs, towards the
construction of a technical system, returning and
impacting the society”. In summary, to design
systems that effectively meet users’ demands, that
understand and respect their culture and values, we
need to see the world through the lenses of these
users, taking into account and articulating the three
levels represented in the Semiotics Onion; we need a
new Science of Design aligning system development
with social practices with the end user.
Besides the Semiotics Onion, the OS theory
provides methods and artifacts, such as the
Stakeholder Identification, Semiotics Ladder and
Ontology Charts, that allow considering the social
world from the articulation of problems stage to the
modelling of computer systems. These methods and
artifacts support designers in understanding the
social world and formalizing it, moving from outside
to inside the Semiotics Onion in order to produce a
computer system. Following we present the VF, an
artifact of OS created for assisting in the
identification and understanding of the cultural (and
social) dimensions of a product (technological or
not) and its impact on people and their values.
3.1 Valuation Framing
Every innovation brings negative and/or positive
impact to the environment in which it is introduced
(Hall, 1959). There are people in that environment
who suffer this impact, trigger others, and confer
values upon such an innovation. Indeed, as Kolkman
(1993) declares, people are always involved and
attaching values to the systems we create because,
otherwise, it would be useless in having these
systems.
Values are defined by Friedman (1996) as
something that a person, or a group of people,
considers important in life; and by Schwartz (2005)
as trans-situational goals that vary in importance and
serve as guiding principles in people’s lives.
According to him, a particular value may be very
important to one person but unimportant to another
because, as Hall (1959) shows, it depends on the
person’s culture, being culturally developed and
negotiated.
A culture consists of many patterns of behaviour
that relate to each other in complex ways. In this
context, each stakeholder group has a cultural
system that governs how it will value an innovation:
different stakeholders may react differently to the
proposed innovation (Liu, 2000). For instance: the
introduction of electronic payment systems through
credit card. The stores, customers, employees,
banks, card management agencies, insurance
companies, IT professionals, and even criminals, are
direct or indirectly interested and/or affected by the
innovation and, consequently, are groups of
stakeholders. These stakeholders may belong to
quite distinct subcultures with different set of values
so that the innovation tends to have rather different
impacts on their lives.
Understanding the potential impact of the
introduction of an innovation, however, requires that
designers are aware of the reactions of these groups
of stakeholders. Kolkman (1993) argues that if an
innovation is inserted in each group accordingly,
probably no serious problems will occur.
Nevertheless, sometimes there may be conflicts and
social world
ICEIS 2011 - 13th International Conference on Enterprise Information Systems
238

designers will be able to anticipate the reactions of
stakeholders only if they could see the world
through the lenses of these stakeholders. The VF
helps in carrying out this kind of analysis by
supporting the identification and understanding of
the cultural dimensions of a product (see Figure 2).
Figure 2: The VF – adapted from (Liu, 2000).
According to Hall (1959), there are 10 areas,
which he calls Primary Message Systems (PMS),
which allow mapping any culture: Interaction,
Association, Learning, Play, Defense, Exploitation,
Temporality, Territoriality, Bisexuality and
Subsistence. The author explains that each culture
develops values in regard to these areas. For
instance, values in bisexuality center around
preferred style of dressing, jobs, sports, and so on, of
men and women. For the VF, Kolkman (1993)
renamed “Defense” to “Protection” and
“Bisexuality” to “Classification”. Indeed, the scope
of Classification goes beyond the notion of gender;
it encompasses issues of age, instructional, social
and economical levels, etc.
The basic principles of the VF are: all the
stakeholders identified in a project are accustomed
to have, in their cultural settings, a range of
behaviour patterns divided into the 10 areas. The
analyst’s work consists of questioning, predicting
and hypothesizing how the innovation can affect/is
affecting these stakeholders regarding these areas.
For instance, in the case of credit card systems, the
stores’ employees (stakeholders) could see the
innovation as a threat in the sense they do not know
how to operate the new machines introduced in their
environment (learning); on the other hand, the
manager could perceive this innovation as an
unnecessary operational cost, once it requires firing
and hiring more employees and/or training them.
The other groups of stakeholders will also value the
innovation from a different perspective. In this
sense, the way we discuss, understand and deal with
the values and cultural systems of each stakeholder
group will determine whether such an innovation
will be appropriated or rejected by them.
According to the exposed, we see the VF as a
powerful artifact for enabling designers to anticipate
and deal with cultural issues in the context of the
project of any innovation. However, using this
artifact is not a trivial activity because it requires
knowledge in anthropology and social sciences. This
knowledge is necessary so that designers are able to
understand the areas that compose a culture and
recognize the values related to each one.
As Sellen et al. (2009) point out, traditionally,
the curricula in Computer Science do not direct
much effort in order to enable its students regarding
social issues. This fact makes it even more important
the research and work with multidisciplinary teams
that can contribute with different visions to a project.
Although a desirable scenario, multidisciplinary
teams are not always possible or viable. In the
example of credit card systems, it would be more
practical for designers to understand some of the 10
areas and their related values because the
stakeholders and the values involved are more
tangible and easy to identify. That is, the problem
space is, at least in parts, well-known to them.
However, in the project of social software it is even
more complicated to know exactly what must be
taken into account. For instance, what are the values
related to the aspects of temporality, territoriality or
association? Also, what values come into play when
the innovation is not a tangible device but a
computer system usable through different objects?
Indeed, regarding social software there are
neither knowledge nor ways (or experience) for
anticipating stakeholders’ reactions, so that dealing
with a so diverse range of stakeholders with quite
different cultural systems become a very costly and
complex task. In these cases, the need for light-
weight and principled methods that support
designers in seeing through the lenses of each
stakeholder group are emphasized. In the next
section we present an effort in this direction: a VF
specially adapted to guide designers in dealing with
values involved in the context of social software.
3.2 A Valuation Framing for Social
Software
The main goal in creating an adapted version of the
VF for the context of social software is to support
designers in the understanding, analysis and
evaluation of such systems. As explained previously,
traditional methods for software evaluation do not
draw attention to cultural aspects and the original
VF is not an easy to use artifact by designers who do
not have experience in social (cultural) issues.
The VF4SS includes an additional column
named “Values” that suggest at least one value for
VALUATION FRAMING FOR SOCIAL SOFTWARE - A Culturally Aware Artifact
239

each one of the 10 anthropological areas (see Figure
3). These values are results of a previous research
which aimed at identifying the values involved in
the context of social software (Pereira et al., 2010).
We must highlight that the list of values is neither
exhaustive nor complete; indeed, our main concern
when creating it was to find a balance between
letting it as comprehensive and diverse as possible
without making it overly complex or detailed.
Figure 3: VF4SS – valuation framing for social software.
To be included into the VF4SS, each value had
to satisfy three conditions: 1. be classifiable into one
of the 10 areas; 2. be discussed without referring to
other values (or areas) and, paradoxically, 3. have
relationships with other values (or areas) influencing
and being influenced by them. These conditions
were inspired in those used by Hall (1959) when
defining the 10 building blocks of culture (areas).
In addition of being classified into the culture’s
areas, each value was also classified through the
Semiotics Onion according to the level that
represents its predominant state. Therefore, values
were classified at the informal (mostly values of
[P]ersonal and ethical nature), formal (collective or
[S]ocial values where there is some rule or system of
norms), or technical level (values that can be
understood as attributes of quality or special features
of [T]echnology). Although this distribution is not
complete for some areas, the spaces corresponding
to the three levels remain explicit in the artifact
(Figure 3) in order to encourage designers to identify
new values and think on the possible manifestations
of each area in each three levels.
Embedded in the original VF, these values favor
designers, evaluators and analysts to keep values in
mind, helping them learn how to use the artifact
itself and situate themselves with respect to what
they must investigate and consider in each area. To
situate our discussions in a practical context, in the
next section we present an experiment in which the
VF4SS was applied to the evaluation of five
different projects.
4 THE CASE STUDY
Aiming at verifying the acceptance and applicability
of the VF4SS, the artifact was used in the evaluation
of five different projects related to the prototyping of
systems for supporting cross-cultural collaboration.
This context was an ideal setting for assessing our
artifact due to the explicit need for dealing with
cultural aspects and, consequently, with values.
These projects were developed in a postgraduate
course called “Topics in User Interfaces: Semiotics
of Human-Digital Artifact Interaction” in which the
Organizational Semiotics theory was used as an
approach for the development of information
systems. The group of participants was formed by 16
designers divided into five groups: G1 (formed by
designers: D1, D2 and D3), G2 (D4, D5, D6), G3
(D7, D8, D9), G4 (D10, D11, D12) and G5 (D13,
D14, D15, D16).
From the five projects for supporting cross-
cultural collaboration, the Project of G1 was related
to sporting events; the Project of G2 and G3 were
related to gastronomy and culinary practices; the
Project of G4 was related to musical tastes and
compositions and the Project of G5 to residential
tourism. This variety was useful because it favoured
the diversity in terms of stakeholder groups,
cultures, values, and also system’s features.
When the evaluation activity started, each group
had completed the documentation and had finished
the prototyping of the first increment of their
systems — see Figure 4 for an example. The main
ICEIS 2011 - 13th International Conference on Enterprise Information Systems
240

goals in evaluating the prototype from designers’
point of view were to identify: i) the impact the
produced system could cause in its different
stakeholder groups; ii) the possible conflicts
between these groups; and iii) the values involved in
the system and the way these values were being
technically supported or promoted. On the other
hand, the two main goals of this activity from our
point of view were: 1. to identify if the VF4SS
would help designers in evaluating social software;
and 2. to verify whether the values suggested would
make sense to designers and what other values
should be included or removed.
Figure 4: Prototypes produced in the Project 1 (G1).
The evaluation activity was carried out as
follows: the identification of all stakeholder groups
involved in the project was already carried out
through the use of another artifact from OS: the
Stakeholder Identification diagram (Liu, 2000). This
artifact distributes the stakeholders in different
categories ranging from the actors directly involved
in the project to the community who will not
necessarily use the system but can be affected by it.
In this context, in the first step designers should
select 3 different stakeholders groups and place each
group in a column of the VF4SS (see Figure 5). In
order to ensure that the system’s cross-cultural
nature was explicit, the groups should be from
different cultures (e.g., Italian, Japanese and
Russian) and from different levels in the Stakeholder
Identification diagram.
In the second step designers should look at the
values suggested in the VF4SS and mark those they
were already considering in their project. In the
third step, designers should analyse and discuss the
importance of each value and the impact it could
cause on each stakeholder group. In the
corresponding cell of the artifact, designers should
indicate how that value was being technically
supported in the project. Finally, in the fourth step
designers should analyse if there would be any
conflict in the way each value was being supported
in the system according to the different stakeholder
groups. If any, they should indicate how the conflict
could be treated.
Figure 5: VF4SS filled in the Project 2 (G2).
As background material for supporting the
evaluation task designers were supplied with: i)
guidelines explaining the four activity’s steps; ii) the
VF4SS both in press and digital format; iii) a
document containing a simplified explanation of
each area; and iv) a table containing a description
and an example for each value suggested in the
artifact. As activity outcomes, each group should
fulfil the VF4SS, answer a survey related to its
applicability, redesign the system according to their
discussions and share their findings with the other
groups.
4.1 Activity’s Main Findings
In general, the evaluation of the projects through the
VF4SS provided us data, insights and evidences that
show the viability of using this artifact for social
software evaluation as well as social software
requirements elicitation and design. Following, we
present some findings and highlight some results
regarding our case study.
From designers’ point of view, the activity
outcomes confirmed our expectations regarding (i)
VF4SS’s usefulness for identifying the impact
caused by the system on its different stakeholders.
All groups reported that VF4SS was determinant in
the process of discussing the challenges, difficulties
and even opportunities for each stakeholder group
regarding the system being prototyped. The VF4SS
and its areas enabled designers seeing (or at least,
trying to see) the system through the lenses of
different stakeholders who would be affected by the
system in different ways. For instance, D10 declared
that “the Valuation Framing brought us [G4] a
VALUATION FRAMING FOR SOCIAL SOFTWARE - A Culturally Aware Artifact
241

better understanding about the impacts that the
introduction of our system could bring to musicians,
producers and fans”. In this project, questions
related to copyright, property and privacy that could
be affected by the system usage were put into scene
by the VF4SS.
Another point also indicated by the VF4SS was
(ii) the identification of possible conflicts between
the stakeholder groups. In some cases the solution to
conflicts was achieved through the specification and
design of other features in the system, while in other
situations it was understood as a new norm, rule, or
even as a system limitation. Some interesting
examples are: “Sponsors want a greater emphasis on
their advertisements, while readers want a clean
interface; Advertisers want to post any content,
while the moderator have to supervise them” (G1);
“A negative rating for a recipe by the users can
bother the system’s sponsor” (G3); “When musicians
are composing a song in a private mode, their fans
should not be able to view it. The system must offer
features that enable them to manage the visibility of
their productions” (G4).
Finally, the VF4SS was also successful in
supporting designers (iii) to identify the values
involved in the project and the way these values
were being technically supported. For instance, in
the Project 2, the VF4SS led designers to think about
the differences in the profile feature according to the
stakeholder group and to redesign the system for
reflecting these differences. In the same project,
designers identified the need for mechanisms to
encourage the participation of users as a way to
technically implement the value of “Emotion and
Affection”. They proposed features that took into
account the different needs and expectations of
stakeholders. For instance, the feature for
encouraging the participation of the “Translator” (of
recipes) was prototyped as a scheme of credits (cash
prizes were cited as an alternative) while the feature
for the “Culinary School” was prototyped as the
possibility for free announcements in the system.
According to designers’ feedback and our own
observations during the execution of the projects, we
could perceive the VF4SS as an artifact capable of
generating fruitful discussions among designers,
allowing them to exercise a critical thinking
regarding the whole impact of the solutions they are
designing. This artifact contributes effectively to the
development of products compatible with the values
of the people they are intended for instead of the
values of their designers. In doing so, it also
contributes to a proper deployment of the product in
the target environment: if a product reflects an
understanding and respect to the values of its
different stakeholders, then, it has better chances of
being appropriated by these stakeholders. These
findings are naturally extended to the original VF.
From the point of view of our research, we
confirmed our hypothesis regarding (1) the utility of
the VF4SS for assisting designers to evaluate social
software, and also regarding (2) the relevance and
benefits of the values suggested in it (i.e., whether
the values suggested would make sense to them).
First, according to the survey designers answered
after the valuation activity, 60% found the values
very useful for the system evaluation; 40% found
them useful; and none answered they are neutral,
unhelpful or useless. According to D4, the suggested
values assisted the group (G2) in carrying out the
evaluation task because they were a starting point.
Because they had no previous experience with
cultural issues, if no values were suggested they
could get lost without knowing what to do or how to
proceed. Therefore, the values suggested in the
VF4SS are important not only to support designers
in carrying out the evaluation of their projects, but
also in learning how to use the artifact itself.
Second, in the survey designers suggested no
additional values to the 28 presented in the VF4SS.
Using designer’s words: “we identified no values to
be included in the framework” (G1); “the table [with
values’ description and examples] is generic enough
for fitting any value into the available options” (G3);
“the suggested values were capable of expressing in
a complete way what we seek and discovered” (G2).
Other evidence that the values suggested into the
VF4SS made sense to designers was the percentage
of values that were effectively considered or
discussed (pointed out as important) in the Projects
— see Table 1. Designers from G3 considered 82%
of the values suggested in the VF4SS but did not
approach new values for their project, while
designers from G1 identified all the values being
expressed in some way. On the other hand, in the
G2, G4 and G5 groups, designers were explicitly
considering 39%, 57% and 79% of the values,
respectively, when the evaluation took place. But,
while filling the VF4SS they recognized the
importance of including new values and discussed
how these values could be technically supported in
their systems.
Table 1: Values considered in each Project.
Group G1 G2 G3 G4 G5
Values considered 100% 39% 82% 57% 79%
Values discussed 100% 61% 82% 61% 100%
ICEIS 2011 - 13th International Conference on Enterprise Information Systems
242

We should highlight, however, that considering
more or less values is not just a question of
designers’ choice but also of the project context and
scope. Consequently, these data do not suggest the
values as a definitive and exhaustive set but that they
made sense to designers, were useful in promoting
critical discussions and met their needs in the
context of their projects.
An interesting example from G2 is related to the
value of “meta-communication” which was not
considered by designers during the system’s
prototyping. However, because the VF4SS
suggested this value in the cultural aspect of
“Learning”, designers started discussing how their
system could technically support it and identified
that each stakeholder group had different views and
different needs regarding this value. For instance,
the stakeholder “Translator” would need support to
understand how the collaborative translation would
work; the stakeholder “Gastronomy school” would
need support to learn how to use the system for
publishing, searching and evaluating recipes; and the
stakeholder “Amateur cook” would need support
through a resource other than text for teaching
him/her how to cook the recipe. Thus, designers
decided to implement the value of meta-
communication through the use of tutorials and
videos placed in the system’s interface; e.g., each
recipe should have a video showing a step-by-step of
how to cook it. After these specifications, the system
documentation was updated and the prototype was
redesigned in order to include the new features.
In fact, the VF4SS not only supported designers
in the task of evaluating the system they were
projecting, but also made they think on new
requirements and features that were missing or could
be included in their systems. By suggesting values,
the VF4SS incited designers to discuss and consider
aspects that were being neglected. Therefore, it
proved to be a useful artifact also for requirements
elicitation. Some feedback from designers confirms
this assertion:
D2: “I would find it very interesting to apply this
artifact [VF4SS] for requirements elicitation. The
reason is quite simple: it enables those involved in
the development process to see, or try to see,
through the eyes of other stakeholders involved in
the project they are proposing. As a developer, I feel
that a lot of rework is caused by developing systems
without thinking of people who will actually use it”;
D3: “The VF4SS is very interesting because it
forces us to imagine the system through the view of
different stakeholders, making designers think
whether the values are being addressed in the
proposed project according to these different
stakeholders. This activity resulted in new
requirements identified. So, in my opinion, it is a
very important activity to be performed at different
times within a project, from requirements elicitation
to the system evaluation”;
D5: “VF4SS is, in my opinion, a great tool not
only for evaluating the design of a system, but also
to identify important requirements”;
D9: “The VF4SS was the tool that I found most
interesting in the whole process. It allows checking
for any conflicting requirements between the various
stakeholders and makes it possible to deal with this
information so that such conflicts do not hinder the
development of the project”;
Grounded on the results briefly discussed in this
section, we are convinced of the viability of using
the VF for the evaluation, and also for the
requirements elicitation, of any technological
artifact. Specifically in the context of social
software, the VF4SS showed to be a promising
artifact for supporting designers in dealing with the
complexity imposed by the social context of these
systems. Indeed, this artifact can be used in research
projects as well as industrial settings favouring
discussions around cultural aspects while guiding
and capacitating designers regarding social subjects.
Finally, this study also contributes to validate the
relevance of the values in the context of social
software we identified in (Pereira et al., 2010).
5 CONCLUSIONS
The design of social software still demand
approaches, artifacts, methods and tools for
reflecting and dealing with the social nature that
characterizes it. In fact, there is even a lack of
theoretically grounded approaches for investigating
this kind of system. Moreover, although clearly
recognized as important, there are few initiatives in
literature related to values in technology. In the
present paper we shed light to this scenario
proposing the VF4SS, an artifact specially adapted
to the context of social software. As a byproduct but
equally important, we introduce and articulate key
concepts and theories, such as the three levels in
which humans operate, the ten basic building blocks
of culture and the Organizational Semiotics theory
with some of its artifacts.
The results obtained from the evaluation of five
prototypes of systems situated in the context of
cross-cultural collaboration indicate the benefits of
using the VF4SS for evaluating as well as designing
social software. Nevertheless, some important points
still remain open and can be seen as a research
agenda in the area. For instance, the VF4SS
VALUATION FRAMING FOR SOCIAL SOFTWARE - A Culturally Aware Artifact
243

produces results essentially qualitative making their
analysis more difficult and their interpretation more
subjective. Although its goal is to bring out aspects
that are difficult to identify and cover areas that
traditionally receive little attention, e.g., values and
culture in technology, studies on possible means of
formalizing and measuring its results are welcome.
Values are intertwined to each other through
complex relationships and these relationships need
to be clarified. Thus, it is difficult to involve values
in the project of technologies if they are considered
in isolation. When considering (or neglecting) a
certain value, other values can be positive or
negatively affected. For instance, depending on the
way the value of meta-communication is being
technically supported in a project, it can affect
differently the value of accessibility, either making it
more difficult (e.g., offering only explanation
through sounds) or promoting it (e.g., offering
multiple media, such as text, images, video, sound).
Consequently, besides the identification of the
relationship among values, if we are to offer
resources for supporting designers to understand and
involve values in their projects, we also need suggest
how these values could be technically supported in
their systems. For instance, autonomy is a critical
value especially in systems related to the exercise of
citizenship, and it has a clear relationship with the
values of accessibility, usability, identity, emotion
and affection, and so on. Mapping this value to a
technical feature is a challenging task not even
always possible.
Finally, although a key artifact, the VF4SS alone
is not enough to guarantee an effective consideration
of values in social software design. Indeed, as the
experiment described in this paper showed, other
artifacts, methods and tools are needed in order to
allow the articulation and involvement of values
during the different stages of a system development
(e.g., the stakeholder identification artifact). We are
naming value-oriented approach (VOA) such set of
tools and artifacts we are investigating in ongoing
and further research.
ACKNOWLEDGEMENTS
This research is partially funded by FAPESP
(#2009/11888-7) and CNPq through the EcoWeb
Project (#560044/2010-0). The authors specially
thank the designers who collaborated with the
evaluation activity and authorized the use of the
documentation of their projects in this paper.
REFERENCES
Ackerman, M. 2000. The intellectual challenge of CSCW:
the gap between social requirements and technical
feasibility. Human-Computer Interaction. v. 15.
Baranauskas, M. C. C. 2009. Socially Aware Computing.
In ICECE’2009 VI International Conference on
Engineering and Computer Education. p. 1-4.
Boyd, D. 2007. The Significance of Social Software.
BlogTalks Reloaded: Social Software Research &
Cases. Norderstedt, p.15-30.
Friedman, B. 1996. Value-Sensitive Design. Interactions.
Nov-Dec. p.16-23.
Friedman, B., Kahn, P. H., Jr., Borning, A. 2006. Value
Sensitive Design and information systems. In Human-
Computer Interaction and Management Information
Systems: Foundations. Armonk. p. 348-372.
Hall, E. T., 1959. The Silent Language. Anchoor Books.
Hendler, J., Shadbolt, N., Hall, W., Berners-Lee, T.
Weitzner, D. 2008. Web science: an interdisciplinary
approach to understanding the web. Communications
of the ACM. Volume 51, July, p. 60-69.
Kolkman M. 1993. Problem articulation methodology.
PhD thesis, University of. Twente, Enschede.
Lazar, J., Preece, J. 2003. Social Considerations in Online
Communities: Usability, Sociability, and Success
Factors. In Cognition in the Digital World. Lawrence
Erlbaum Associates. p. 127-151.
Liu, K. 2000. Semiotics in Information Systems
Engineering. 1st edition. Cambridge University Press.
Miller, J., Friedman, B., Jancke, G., Gill, B., 2007. Values
Tensions in Design. In ACM GROUP’07, Florida. p.
281-209.
Neris, V. P. A., Martins, M. C., Prado, M. E. B., Hayashi,
E. C. S., Baranauskas, M. C. C. 2009. Design de
Interfaces para Todos-Demandas da Diversidade
Cultural e Social. In XXXV Semish/XXVIII Brazilian
Computer Society Conference. p. 76-90.
Pereira, R., Baranauskas, M. C. C. & Silva, S. R. P. 2010.
Softwares Sociais: Uma Visao Orientada a Valores. In
IX Brazilian Symposium on Human Factors in
Computer Systems ( IHC’10). ACM. p. 149-158.
Schwartz, S. H. 2005. Basic human values: Their content
and structure across countries. In Values and
Behaviors in Organizations. Vozes. p. 21-55.
Sellen, A; Rogers, Y.; Harper, R. & Rodden, T. 2009.
Reflecting Human Values in the Digital Age.
Communications of the ACM. Vol 52. p. 58-66.
Smith, G., 2007. Social Software Building Blocks.
http://nform.ca/publications/social-software-building-
block. /, [Feb, 02, 2011].
Thompson, A-J. & Kemp E.A. 2009. Web 2.0: extending
the framework for heuristic evaluation. In ACM
CHINZ 2009. p.29-36.
Webb, M. 2004. On social software. http://interconnected.
org/home/2004/04/28/on_social_software, [Feb, 02,
2011].
ICEIS 2011 - 13th International Conference on Enterprise Information Systems
244