Development of a Knowledge Base for Social Experimentation
Steven B. Kraines
Future Center Initiative, the University of Tokyo, Kashiwa-No-Ha, Kashiwa-Shi, Japan
Keywords: Social Experimentation, Metadata and Structured Documents, Tools and Technology for Knowledge
Management, Knowledge Management Projects, KM Strategies and Implementations.
Abstract: Social experimentation could be useful for testing the feasibility and effectiveness of technologies and
policies in achieving more sustainable social systems. However, classical social experiments are costly and
can only be applied in a limited range of situations due to the requirement for randomized assignment of
subjects to experimental and control groups. Here, we reconsider the role of social experimentation within
the framework of the feasibility of technology and policy interventions for creating societies that are more
sustainable, particularly in regard to mitigation of CO
2
emissions and aging populations. From a review of
more than 100 social experiments from the literature, we develop a knowledge schema and knowledge base
system for structuring and managing the valuable knowledge that has been produced under the scientific
theme of social experimentation. The knowledge base contains classical randomized social experiments, but
it also includes studies that are less rigorous from the point of view of random assignment.
1 INTRODUCTION
Meeting future global challenges requires
coordinated efforts of a wide range of knowledge
experts and social actors to establish new technology
and policy interventions that enable societies to
mitigate and/or adapt (Takeuchi and Komiyama
2006). Interventions having the greatest potential for
increasing the sustainability of a region must be
identified and decision-makers informed as to how
to get those interventions accepted by society.
Scientific evidence regarding the feasibility and
effectiveness of different interventions must be
managed effectively. We consider the feasibility of
technology or policy interventions at four levels:
theoretical, technological, economic, and social.
Economic feasibility concerns “external” barriers
to technology adoption such as legal mechanisms,
support infrastructure and supply channels. Social
feasibility addresses “internal” barriers (McKenzie-
Mohr 1996). Even if all of the external barriers are
overcome, barriers associated with consumer values
and perceptions, information transparency, and the
agenda of major stakeholders can still prevent
successful introduction. Furthermore, although
external barriers can usually be addressed by models
which assume that all participants will act rationally,
internal barriers often involve irrational aspects of
human behaviour and decision-making. While
internal barriers can be externalized, for example by
offering economic subsidies, often an internal barrier
can defeat externally driven attempts to get the
technology adopted (McKenzie-Mohr 1996).
Of the four levels of feasibility, social feasibility
usually has the highest dependency on context.
Measures to overcome internal social barriers
require changes in the attitudes and behaviours of
local stakeholders, which can differ greatly in
different contextual settings. Technologies for
sustainable societies are particularly sensitive to
context because of the large number and diversity of
stakeholders (McKenzie-Mohr 1996). Due to this
contextual dependency, even if an intervention is
demonstrated to be effective in one context,
additional social feasibility studies will often be
necessary when considering it for another context.
Social experimentation, which was developed as
a technique for providing information that helps
policy makers make decisions about public policy
(Orr 1998), is a powerful tool for assessing the
social feasibility of a proposed intervention. Social
experiments are used to 1) “influence specific policy
decisions or address unresolved scientific or
intellectual issues;” 2) “influence core policy
decisions or general intellectual orientations;” 3)
“influence relatively narrow elements of policy and
164
B. Kraines S..
Development of a Knowledge Base for Social Experimentation.
DOI: 10.5220/0004134201640167
In Proceedings of the International Conference on Knowledge Management and Information Sharing (KMIS-2012), pages 164-167
ISBN: 978-989-8565-31-0
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
its implementation” 4) offer pre- and post- decision
support (Greenberg and Shroder 1997).
In classical social experimentation, “selection
bias” is avoided by random assignment of people
from a target population to two groups: an
experimental group to which the intervention is
applied, and a control group which experiences
conditions identical to the experimental group
except for the intervention. This is the only way to
guarantee that there is no systematic difference
between those participants who are subjected to the
intervention and those who are not (Orr 1998).
However, randomized social experiments are
costly and difficult. As policy makers face the need
to rapidly make decisions about social interventions,
“social experimentation” has come to be understood
to mean “‘trying out’ a new program on a small
scale, to see if it ‘works’”, e.g. in the form of a pilot
demonstration (Orr 1998). Although the scientific
validity of such non-randomized studies may be
suspect, given that the role of social experimentation
is to provide policy makers with information on
upon which to base public policy, it is worthwhile to
re-examine the conditions under which a study could
be considered to be a “social experiment”.
A major concern in Japanese public policy is
how to design societies with low carbon emissions
that still support “successful aging” of Japan’s
rapidly aging society (Platinum Concept Network
2012; Bright Low Carbon Society 2012). Japanese
policy makers could benefit from social experiments
that provide useful knowledge on the effectiveness
and costs of different interventions for removing
internal barriers to technologies and policies aimed
at improving the health and quality of life for elderly
people while simultaneously reducing energy
consumption or switching to low carbon energy in a
target urban region or for a target population.
The expected value of a social experiment is the
value of a change in policy times the probability of
that change occurring as a result of the experiment
minus the experiment cost (Orr 1998). Because the
realization of a policy change depends on many
complex factors, the use of random sampling to
eliminate selection bias may be less important than
other factors in whether or not a social experiment
will influence public policy. Orr (1998) notes that
“even if one cannot confidently assert that the
treatment-control service differential represents the
service increment that would result from adoption of
the program, the impact estimates based on that
differential may still…provide valid estimates of the
effects of a wellspecified policy change.”
Here, we have reviewed over 100 studies of
social interventions in the areas of aging
populations, energy consumption and environmental
behaviour that fit the following minimum
specification: “some experience-based knowledge
about the effect of some technology and/or policy
combination on changing some aspect (e.g.
population behaviour) of a target region so as to
improve some combination of the targeted
conditions.”
2 SCHEMA DEVELOPMENT
The studies that we examined reveal a tension
between traditional social scientists interested in
developing a statistically rigorous body of
knowledge on factors controlling human behaviour
and more problem-driven studies such as action
research. We observed a tradeoff between several
dimensions of experiment design, listed below.
representativeness of sample population
adequate sample size for statistical validity
adequate temporal length / followup studies,
controlling for confounding factors
use of well-established, quantitative measures
useful and relevant intervention designs
provenance issues, replicability
To examine the variation of the studies in more
detail, we developed a schema for social
experiments based earlier reviews (Greenberg and
Shroder 1997, Abrahamse et al 2005, Cattan et al.
2005, Dwyer et al. 1993, Hogan et al. 2002). The
following questions motivated schema development:
What is the proposed intervention?
Who (and where) does the intervention target?
What change (e.g. in behaviour) is intended?
Who are the major social actors involved?
What does the intervention cost to implement?
Who/what was actually studied?
What are the main results and future topics?
Are there reusable project deliverables?
For each study, we created entries for the fields in
the schema that were applicable to that study. The
complete schema with totals for how many studies
had entries for each field is shown in Table 1.
The reasoning capability of the knowledge base
is realized by grounding the entries for each field in
the schema to a “heavy weight” ontology based on
description logics that supports semantic reasoning
based on logical inference (Guo and Kraines 2008).
DevelopmentofaKnowledgeBaseforSocialExperimentation
165
Table 1: The proposed schema and its usage in the knowledge base for social experimentation.
Target Population: the population that the intervention is intended to target
demographics (147); location (124); recruitment method (87);
living condition (57), job type (24); education Level (21); disabilities (28)
Study Goal: major search condition
hypothesis (81); outcomes of interest (193); targeted behaviour (135); problem addressed (86)
Intervention: major search result
cost of economic interventions (32); tech type used (105); group/individual (134);
number and duration of sessions (62); issues (54); theoretical basis (87)
Study Setting: of the actual social experiment or study
duration (111); start & end dates (101); researchers (232); study location (170);
study cost (8); funding source (91); study resources (16)
Study Groups: type of participants (115); group sizes (113); control? (132); random assignment?(109);
specific activities (126); intervention types (140)
Measurements: assessment method (67); timing (27); scales/instruments used (51); follow-up (78)
Results: summary of effect (116); time trends (34); quantitative measurements (38);statistical significance (47);
generalizability (64); replicability (52); limitations (40); design issues (36)
Deliverables: databases, techs, software (1, 15, 10); Future topics (45)
Figure 1: Semantic graph representation of the
intervention details for the social experiment “Social
Activation of the Elderly: a Social Experiment”
b
y Arnetz
et al. 1982.
Figure 2: Natural language generation results in Englis
h
for the semantic graph in figure 1.
We extract noun and verb phrases from each of the
field entries, and then we normalize the terms to
classes from the SCINTENG ontology (Kraines and
Guo 2011) augmented with the terminology from the
WHO international classification of functioning,
disability and health. Term normalization was done
by mapping the ontology to WordNet via the SUMO
ontology. Finally, we generate semantic graphs
representing the knowledge in the schema fields in a
form that can be “understood” by a computer. The
graph generated for the intervention details of an
entry in the knowledge base is shown in figure 1.
We consider three kinds of users for the
knowledge base. People studying the effectiveness
of interventions in meeting some societal need can
examine previous social experiments studying
similar interventions and/or similar societal needs
could produce valuable hints. Local governments
can find information on interventions to help address
problems in the governed region from social
experiments used in similar locations. Funding
programs and national policy development can use
the structured knowledge base as a map of the
existing scientific knowledge for achieving a
sustainable society, e.g. to identify "knowledge
gaps" requiring more research support. The
knowledge base could also guide a process for
building and reviewing empirical evidence for a
particular intervention, e.g. prior to providing "social
venture capital" or a regional contract by a local
government (see for example Orr 1998 Part 7).
Consider the following use scenario. A
researcher wants to study an intervention that is
produced in part by the people who are the targets of
the intervention. This search condition could be
described using the following SPARQL query:
KMIS2012-InternationalConferenceonKnowledgeManagementandInformationSharing
166
select distinct ?s1 where {
?s1 a <#policy_artifact> ?s2 a <#person> .
?s3 a <#human_activity> ?s4 a <#planning> .
?s1 <#related_to> ?s3 .
?s3 <#has_participant> ?s2 .
?s4 <#has_participant> ?s2 .
?s4 <#produces> ?s1
}
A description logics reasoner could then determine
that this query matches the semantic graph shown in
figure 1 because “Socializing” is described in the
ontology to be a subclass of “human activity”.
Consider adding a query constraint specifying
that the persons involved in both the planning and
the activities of the intervention were the focus of
the problem type addressed by the intervention.
Although the problem type “need for engaging the
staff” exists in the graph, it applies to persons
involved in the planning of the intervention but not
in the activities of the intervention. Thus, the query
would no longer match with the semantic graph.
3 FUTURE DIRECTIONS
Future work on this knowledge base will focus on
developing effective applications in areas that are
most likely to be beneficial to the envisaged users.
Natural language generation can be used to generate
accurate representations of the semantic graphs in
any language that is handled by the generator, as
shown in figure 2. Knowledge mining techniques
can be used to extract common semantic motifs from
the knowledge base on what kinds of interventions
are most effective in what kinds of social contexts.
Obtaining feedback from potential users of the
knowledge base will be critical in guiding the
development of these applications. For this purpose,
we hope to establish a “community of practice”
through the Platinum Concept Network in Japan and
other entities around the world, such as the United
Nations HABITAT program (www.unhabitat.org).
Feedback from these test users will also help to
identify what modifications to the knowledge base
schema are needed. Under the theme of “social
entrepreneurship”, a new class of NPOs is emerging
that shares some of the flexibility for experimental
trial-and-error provided by venture capitalists
(Tanimoto 2008). These NPOs may be valuable
sources for knowledge on what works in addressing
specific social issues in specific social contexts.
System development is focused in the human-
computer interface. We are testing different methods
for accessing the knowledge base, one of which is
based on semantic similarity calculated between
cities using semantic attributes from DBpedia (Guo
and Kraines 2010). We are also using knowledge
mining and natural language processing techniques
to further assist researchers and social entrepreneurs
to create semantic graphs that accurately express the
knowledge that they want to share.
REFERENCES
Abrahamse, W., Wokje, A., Steg, L., Vlek, C., and
Rothengatter, T. 2005. A review of intervention
studies aimed at household energy conservation.
Journal of Environ Psychology 25(3): pp. 273-291.
Cattan, M., White, M., Bond, J. and Learmouth, A. 2005.
Preventing social isolation and loneliness among older
people: a systematic review of health promotion
interventions. Ageing & Society 25: pp. 41-67.
Dwyer, W. O., Leeming, F.C., Cobern, M.K., Porter, B.E.,
and Jackson, J. M. 1993. Critical review of behavioral
interventions to preserve the environment: Research
since 1980. Environ and Behavior 25: pp. 485-505.
Greenberg, D. H. and Shroder. M. 2004. Digest of Social
Experiments Third Edition. The Urban Institute Press.
Guo, W. and Kraines S.B., 2008. Explicit scientific
knowledge comparison based on semantic description
matching. ASIST 2008, Columbus, Ohio.
Guo W., S. B. Kraines. 2010. Enriching city entities in the
EKOSS failure cases knowledge base with linked open
data. Proc 6th Intl Conf on Next Gen Web Services
Practices, Nov 23-25, 2010, Gwalior, India, pp: 58-63
Hogan, B. E., Linden, W., and Najarian, B. 2002. Social
support interventions Do they work? Clinical
Psychology Review 22: pp. 381-440.
Kraines, S. B. and Guo, W., 2011. A system for ontology-
based sharing of expert knowledge in sustainability
science. Data Science Journal, 9: 107-123.
McKenzie-Mohr, D. 1996. Promoting a Sustainable
Future: An Introduction to Community-based Social
Marketing. http://nrtee-trnee.ca/wp-content/uploads/
2011/08/promoting-sustainable-future.pdf
Orr, L. L. 1998. Social experimentation: evaluating public
programs with experimental methods. Office of the
Assistant Secretary for Planning and Evaluation, U.S.
Department of Health and Human Services.
Platinum Concept Network. 2012. www.platinum-
network.jp (Japanese only)
Tanimoto, K. 2008. A Conceptual Framework of Social
Entrepreneurship and Social Innovation Cluster: A
Preliminary Study. Hitotsubashi Journal of Commerce
and Management, 42(1): pp. 1-16.
Takeuchi, K. and Komiyama, H., 2006. Sustainability
science: building a new discipline. Sust Sci, 1(1): 1-6.
Urban Reformation Program for Realization of Bright
Low Carbon Society. 2012. http://low-carbon.k.u-
tokyo.ac.jp/en/index.html
DevelopmentofaKnowledgeBaseforSocialExperimentation
167
