EFFECTIVE POLICY BASED MANAGEMENT OF 3D MULE
An Exploratory Study Towards Developing Student Supportive Policy
Considerations
Indika Perera, Colin Allison and Alan Miller
School of Computer Science, University of St. Andrews, Fife, KY16 9SX, Scotland, U.K.
Keywords: 3D Multi User Learning Environments, Policy based Management, Second Life, Open Simulator,
Self-regulatory Learning, Learning Environment Management, Student Engagement.
Abstract: Learning environments that are based on 3D Multi User Virtual Environments (3D MUVE) can be referred
as 3D Multi User Learning Environments (3D MULE). 3D MULE used in various educational and research
activities show proven success sufficient to warrant their consideration as a mainstream educational
paradigm. They introduce a platform for diverse learning activities with a novel set of challenges for
teachers and students. Without suitable learning management practices, 3D MULE users can encounter
difficulties during their learning interactions through 3D MUVE functions, although the learning
environment is dynamic and engaging. To overcome this challenge, we researched learner supportive policy
considerations for 3D MULE management. This paper presents an exploratory study with student
engagements (N=32) that identifies key factors, self-regulation and environment management, for policy
considerations. Moreover, the paper critically examines the importance of constructive alignment of
learning activities with the 3D MULE features for useful learning experiences.
1 INTRODUCTION
3D virtual worlds provide unique features for
enhancing technology supported learning through
intuitive activities for learning complex and
advanced concepts. They are particularly appropriate
for educational use due to their alignment with the
Kolb's (Kolb et al., 2001) concept of experiential
learning, and learning through experimentation as a
particular form of exploration (Allison et al., 2008).
For our research we use Second Life (Linden Labs,
2003) and Open Simulator (2007) MUVE; a trend
towards open 3D MUVE such as OpenSim can be
seen recently (Allison et al., 2011), however.
Lack of awareness on 3D MUVE system
functions can cause significant challenges of
learning management for an educationalist, which
can deteriorate the value of 3D MULE and student
motivation for learning. Furthermore, if adequate
management policies are not followed, students may
also find difficulties in adhering to the best practices
when engaged in learning. Managing 3D MULE and
achieving usability and trust in learning can be a
challenge as student interactions may be influenced
by the varying levels of self-regulation practices.
With the interest for extensive use of 3D MULE, we
believe that the use of learner supportive
management policies would positively contribute to
student learning engagement. Therefore, we have
focused on identifying key policy areas for
managing 3D MULE.
Section 2 of this paper describes background
details and our experiences with 3D MULE; section
3 reveals the research methodology and design.
Sections 4 and 5 elaborate the results, analysis,
contributions and study limitations. Section 6
presents future work and conclusions.
2 BACKGROUND AND RELATED
WORK
Interactive 3D virtual environments demonstrate a
great educational potential due to their ability to
engage learners in the exploration, construction and
manipulation of virtual objects, structures and
metaphorical representations of ideas Dalgarno (et
al. 2009). Although the learning experience could be
implemented on other platforms without 3D MUVE
support, the learner experience would be lost, and
193
Perera I., Allison C. and Miller A..
EFFECTIVE POLICY BASED MANAGEMENT OF 3D MULE - An Exploratory Study Towards Developing Student Supportive Policy Considerations .
DOI: 10.5220/0003921101930199
In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 193-199
ISBN: 978-989-8565-06-8
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
users feel contrived (Girvan & Savage, 2010).
Various successful studies for enhancing education
with 3D MULE can be found, recently. However,
most of these studies assume the fact that 3D MUVE
implicitly facilitate learning needs. Since 3D MUVE
are not specifically designed for educational needs,
users have to consider system and user role based
management for successful learning experience
(Perera et al., 2011). Weippl (2005) has considered a
set of factors for e-Learning system management
policy considerations. Previous study on use case
analysis of 3D MULE compared to e-Learning has
identified the significance of environment
management for successful management policy
considerations (Perera et al., 2011).
Technology supported learning environments,
can help to develop specific self-regulatory skills
related to successful engagement in learning
(Dabbagh & Kitsantas, 2004). Pintrich, (2000)
defines self-regulation as “an active, constructive
process whereby learners set goals for their learning
and then attempt to monitor, regulate, and control
their cognition, motivation, and behaviour, guided
and constrained by their goals and the learning
environment contextual features”. Students with
better self-regulatory skills tend to be more
academically motivated and display better learning
(Pintrich, 2003). Schunk (2005) suggests the need of
more research aimed at improving students' self-
regulatory skills as they are engaged in learning and
to examine how learning environment contexts
affect the amount and type of self-regulation
displayed. In this study, we examine self-regulation
and system management for 3D MULE learning.
Several research and educational projects with
3D MUVE are used in the University of St Andrews
as 3D MULE such as, The Laconia Acropolis
Virtual Archaeology (LAVA) (Getchell et al., 2010),
Wireless Island (Sturgeon et al., 2009), Network
Island (McCaffery et al., 2011) and Human
Computer Interaction (HCI) student projects (Perera
et al., 2009). Research on integrating 3D MUVE
with e-Learning infrastructure is conducted (Perera
et al., 2011), which initiated this study.
3 RESEARCH METHODOLOGY
AND EXPERIMENT DESIGN
Student behaviour and system administration have
been widely considered aspects for effective
learning environment design. Based on our previous
studies on 3D MUVE use cases and role analysis
(Perera et al., 2011) and the 3D MUVE function
behaviours, we hypothesised these two parameters to
be the most influential factors for successful policy
considerations for 3D MULE management.
Furthermore, we decided to extend the study to
investigate the impact of these two factors on
student engagement with the 3D MULE. Therefore,
we examine three factors: student behaviour, system
(environment) management and student engagement
with the environment as the research variables.
Importantly, the engagement with 3D MULE may
not necessarily represent the student engagement
with the learning, although there can be a positive
correlation if the learning tasks are constructively
aligned (Biggs, 1996). However, the opposite; i.e., if
the student engagement with the 3D MULE is low,
so it is with the learning tasks that depend on 3D
MULE, obviously. There are unique advantages of
using 3D MUVE for teaching and learner support,
which we may not achieve from the other methods.
Deductively, if students do not engage with 3D
MULE, there is a high tendency of them having less
engagement with their learning as well. Hence, our
policy considerations for 3D MULE management
should not negatively affect the student engagement.
The following research hypotheses were defined
to examine the supposed variables and their impact.
H1: Student behaviour with self-regulation is a
major factor of the successful 3D MULE
management
H2: System environment management is a major
factor of the success of 3D MULE management
H3: Student self-regulatory behaviour has a
positive and significant effect on student
engagement with 3D MULE
H4: System environment management has a
positive and significant effect on student
engagement with 3D MULE
3.1 Experiment Design
We followed a two phase study; first, students from
two course modules experienced the 3D MUVE
supported learning environment (in OpenSim) as a
part of their studies. In contrast to Second Life,
OpenSim gives several advantages (Allison et al,
2010). It is important that students participate in a
credit bearing learning session designed with 3D
MUVE for the data accuracy than following an
artificial task. Further, it helps students to
consciously associate the experience they had. The
two modules have different learning objectives
levels in Scottish Credits and Qualification
Framework (SCQF, 2007). This study focused on
student engagement with the environment; although
CSEDU2012-4thInternationalConferenceonComputerSupportedEducation
194
students had different levels of the same learning
task, both samples were similar with respect to our
measures. The differences in the modules did not
affect the experiment; hence can be considered as a
single sample of 59 students for the data analysis.
Table 1: Considered module information.
Module Information SCQF Level
CS3102 Data communication and Networks
- 31 Students
10- Hons.
Degree
CS5021 Advanced Networks and Distributed
Systems - 28 Students
11-PG
Degree(MSc)
The learning environment, Wireless Island
(Sturgeon et al., 2009), is a dedicated region for
facilitating learning and teaching wireless
communication through interactive simulations with
configuration settings and supplementary learning
content for exploration. To facilitate small group
learning with a less competitive environment
interaction, we set up 5 identical Wireless Islands
(256x256 m
2
virtual space; 6 students per island) in
the OpenSim environment. Students were assigned
to regions as their home place to start their learning.
For the second phase, a questionnaire with 15
questions, divided into two sections: avatar
behaviour–7 questions, and 3D MULE management
–8 questions, was used. The 8 questions in the 3D
MULE management section had some relevance on
the two factors that we are investigating, self-
regulation and environment management; however,
the questions did not directly portray the variables.
We decided to confirm through the statistical
analysis, therefore, included as 8 related questions.
4 RESULTS AND ANALYSIS
4.1 Observed Student Engagements
Avatar appearance change can be a fun, although
creating attractive appearances within a limited time
can be challenging. OpenSim with Hippo viewer
gave an additional step on changing user appearance
compared to Second Life. Users first have to create a
body-part, edit it and then wear it to change the
shape of the default avatar. However, few students
spent long enough to create more sophisticated
shapes, clothes and even change the avatar gender
(default avatar shape is shown in the top-left image
of Fig. 1). Postgraduate students were less keen on
changing their avatar, whereas many undergraduates
went to a further step by comparing the avatar
appearances with their friends’. However, the
student commitment for making their avatars look
good should not be underestimated as it can take
substantial portions of their learning time.
Content creation is one of the fundamental
interactivity mechanisms available in 3D MUVE.
Allowing students to create attractive 3D shapes
makes them engaged with the environment
passionately. As observed, students tried a range of
constructions and editing of the existing objects.
Figure 1: Observed student interactions.
Some of the alterations directly affected the
learning experience; activities such as wearing the
control buttons of the media displays, moving and
changing the internal arrangement of the lecture
theatre, and creating constructs on the simulation
area (Fig. 1), introduced some disturbances. These
can be discouraged through usable policy
considerations. A student interaction caused the
learning environment to be significantly altered
compared to its original layout (Fig.1 last row). This
observation was a one-off incident, as the majority
of students refrained from changing land settings.
Compared to the postgraduate (Masters) students,
the undergraduate (Honours) students showed high
interactivity, resulted in a range of user-created
objects, altered content and changed land terrain.
The undergraduates were keen on exploring game-
like features, and associate their friends for
collaborative activities, although those activities
were not related with the learning. Students that
were keen on completing their tasks may have had
less motivation to explore the 3D MUVE, however.
Students were allowed to follow their preferred
behaviour and environment interactions as a mean of
learning through exploration (Kolb et al., 2001)
without any restriction. In this exploratory study, we
wanted to examine these actions as empirical
EFFECTIVEPOLICYBASEDMANAGEMENTOF3DMULE-AnExploratoryStudyTowardsDevelopingStudent
SupportivePolicyConsiderations
195
evidence; an assurance was given that their
behaviour does not affect their grades, but the
completion of the learning tasks.
4.2 Analysis of Questionnaire Data
We received 32 completed questionnaires (54.28%),
[20 (71.4%) from postgraduate and 12 (38.7%) from
undergraduate students]. An initial observation of
the question characteristics and the descriptive
statistics helped to understand the user responds and
possible classifications. Analysis of the question and
responds resulted in preliminary clustering of the
questions into two major categories: (Q1-Q7) user
engagement and (Q8-Q15) 3D MULE management.
The questions in user engagement section, based
on different student behavioural activities in the
environment, indicated a higher internal consistency
(Cronbach’s α = 0.802). The internal consistency
validated the combined use of questions to represent
the associated variable. The 3D MULE management
section was designed to examine the two prime
variables associating the research hypotheses. We
have statistically analysed for accurate variable
identification as a method to test the hypotheses.
Exploratory Factor Analysis (EFA) used to test
the Q8-Q15 question set. Pre-tests were conducted
for the fitness of data to be used for EFA.
Bartlett
Test of Sphericity, a strict test on sampling and
suitable correlations, was performed using PASW
(18.0), and obtained χ
2
=155.257, p<.001, suggesting
that the correlation matrix (R-Matrix) items can be
clustered based on relationships; the null hypothesis
of R-Matrix being an Identity Matrix can be rejected
with significance indicating a higher level of fitness
for EFA.
Additionally, the Kaiser-Meyer-Olkin
(KMO) test to examine the accuracy of using the
data sample for the EFA; KMO value =0.714(>0.7;
Field, 2006) validated the fitness. The sample size to
variable, N:p was higher and meets the guidelines
given by Costello & Osborne (2005).
We used EFA to test H1 and H2. Principal
Component Analysis (PCA) was used to extract
factors; two factors (Eigenvalues>1.0) was obtained.
These highest two factors contribute nearly 72% of
the total variation of the aspect 3D MULE
Management.
Further, we employed Orthogonal
Verimax Rotation with Kaizer Normalization and
obtained the rotated factor loadings (Fig. 2).
To remove weak loadings, we followed Stevens
(1992) and Field (2006) suggestions, and used .6 as
the cut-off, considering the exploratory nature of the
analysis. The Orthogonal Verimax Rotation seems
accurate as the two factors relatively equally
contribute (37.13% and 34.74%) to the underlying
aspect. The Component Transformation Matrix
showed symmetry over the diagonal, indicating the
Orthogonal Rotation is accurate, and the rotated
factor loadings are correct. As the rotated factor
loadings indicated, questions Q9, Q15, Q10 & Q8
were considered as a one variable. The objectives of
the questions suggested a common parameter. We
concluded it as the student behaviour with Self-
regulation, as expected. The second factor represents
Q11, Q14, Q12 & Q13 questions and strongly
relates the student preference and impact on the
system control, administration and management of
Table 2: Questionnaire items and the descriptive statistics on item scores.
No Question Mean Mode Std. Dev. Std. Error
Avatar Engagement
Q1 I changed my appearance as I like to appear 3.09 3 0.466 0.082
Q2 I created content objects in the environment 3.69 4 0.471 0.083
Q3 I tried to change the land or content objects in the learning environment 3.66 4 0.483 0.085
Q4 I communicated with others regularly
3.53 4 0.507 0.090
Q5 I have followed other avatars collaboratively during my learning 3.44 3 0.504 0.089
Q6 I moved to all the places in my island and teleported to other islands 3.81 4 0.397 0.070
Q7 My activities in the environment resulted in a high engagement with my learning
tasks
3.97 4 0.40 0.071
Self-Regulation
Q8 I think my behaviour affected others’ learning 3.31 3 0.592 0.104
Q9 The open space and other avatars made me to interact as in a real-world learning
session
4.05 4 0.354 0.064
Q10 Use of real identities increases the proper behaviour of students 4.02 4 0.309 0.056
Q15 Students should responsibly use the learning environment 4.10 4 0.296 0.051
Environment Management
Q11 Land and Content management controls are important to manage environment 3.78 4 0.420 0.074
Q12 System control and management practices are important for a reliable learning 3.91 4 0.296 0.052
Q13 System management settings should not reduce the 3D MUVE usability 4.19 4 0.592 0.105
Q14 Appropriate system security & controls ensure a successful learning experience 3.89 4 0.390 0.070
CSEDU2012-4thInternationalConferenceonComputerSupportedEducation
196
Figure 2: Rotated Component Matrix and Component Plot
in Rotated Space.
the 3D MUVE. Therefore, we defined this variable
as Environment Management to cover all the aspects
that associate. The internal consistencies among the
question items within the three variables found to be
very high (Cronbach α >.8). The items on self-
regulation and environment management also meet
the requirements to be considered collectively to
represent the variables. One-Sample Kolmogorov-
Smirnov test for normality on the variables student
engagement, self-regulation, environment
management resulted in [X~N(3.598,0.283),
α=0.997], [X~N(3.876,0.378), α=0.516],
[X~N(3.95,0.175), α=0.796], respectively, retaining
the hypothesis of normal distribution confirming the
collective use of items with normality.
Table 3: Regression model summary.
Variable β Std. Err. t-value Sig.
Self-Regulation .240 .097 2.482 0.019*
Environment
Management
.657 .092 7.312 0.000**
* p < 0.05, ** p < 0.001
The Spearman rho between Self-Regulation and
Environment Management is 0.398 indicating a
weak positive relationship (p<0.05). This is
important since the two variables measure
sufficiently different parameters and the inter-
relationship is insignificant. Therefore, we conclude
that, Self-Regulation and Environment Management
are sufficiently independent as measuring two
different aspects, which further proves the EFA and
selection of variables to represent the 3D MULE
Management aspect.
Moreover, there aren’t any
other strong and significant variable revealed
through the EFA. Therefore, the research hypothesis
H1 and H2 are substantiated. To test the research
hypotheses H3 and H4, we used regression analysis.
A sample size test was done for the fitness for
regression analysis. As in Field (2006), for the test
statistics of anticipated large effect (F
2
=0.35),
Number of predictors (n=2), Probability level of
Significance (α=0.05) with the desired statistical
Power level of (1-β=0.8), the minimum required
sample size was 31. Therefore, our sample size
N=32 (>31) suites well for the analysis and the
regression model is reliable.
Linear regression analysis, R
2
=0.759, indicates
that about 75.9% of the variation in the student
engagement is determined by the environment
management and student self-regulation in 3D
MULE, as a combined effect. ANOVA of the model
fit showed, that the regression model significantly
explains the Student Engagement from the variables
Environment Management and Self-Regulation
(p<0.001). As the variable relationship with
predictor parameters of the model shown in table 3,
the path coefficients are .240 for Self-Regulation,
which is significant (p<0.05) and .657 for
Environment Management with significance
(p<.001). Therefore, the research model
substantiates our hypotheses H3 & H4.
5 DISCUSSION
Available space prevents a more comprehensive
discussion on the individual questions and the
recorded scores; the question responses are self-
explanatory with the results shown in Table2.
However, briefly, Q7 suggests an important
relationship between the environment engagement
and learning engagement, which is highly
recommended for further study. The Q5 and Q8
show the importance of associating collaborative
learning tasks with the available 3D MUVE
facilities. Student collaboration occurs through
learner interaction while interacting with the 3D
MUVE that provides supports rather than barriers to
learning (Girvan & Savage, 2010).
EFA and consistency tests resulted in the
identification of the two variables: self-regulation
and system environment management as the major
factors of 3D MULE management, confirming the
hypotheses H1 and H2. Therefore, we suggest these
two parameters as main consideration areas for 3D
MULE management policy development. With
reference to H3 and H4, the student self-regulation
on learning activities and 3D MUVE management
practices result in a significant positive effect on the
student engagement with the environment and
learning tasks. We conclude that, for constructive
EFFECTIVEPOLICYBASEDMANAGEMENTOF3DMULE-AnExploratoryStudyTowardsDevelopingStudent
SupportivePolicyConsiderations
197
and successful 3D MULE student engagement, we
must identify and implement policies for student
self-regulation and 3D MUVE system management.
Thus, we validate our research hypotheses on
considering self-regulation and environment
management of 3D MUVE as a prime factor for 3D
MULE management policy considerations.
The environment management indicated the
highest positive impact on increasing the student
engagement. Although, students entertain
themselves by various environment engagements
they also felt the difficulty of task coordination and
unsupportive avatar behaviour during learning
engagements. For example, lecture displays reset
whenever an avatar hits the play button, disturbing
the other viewers. Also, if a student’s simulation
arrangement is too close to another’s setup,
simulation interferences were observed. These can
be easily solved if we implement suitable
environment management policy considerations.
Considering the student opinions and our
observations, we suggest the importance of cohesive
learning activities with 3D MULE user engagement
aspects through constructive alignment (Biggs,
1996). This would enable students to spend their
time highly engaged with the 3D MULE, while
achieving their learning goals comfortably.
Careful analyses and tests were employed to
minimise the impact of the following limitations.
Due to the nature of the research, the study sample
was limited to a particular set of students. These
students have provided their feedback and answers
based on their experiences, which we validated
through observation. Therefore, we consider the data
received as accurate and conclusive though the study
had a relatively small sample, due to resource
constraints. The questions used were appropriately
designed, although they have yet to be examined for
psychometric measures. It is a challenge to find a
widely accepted standard set of psychometric
measures in particular for 3D MULE, as the field of
study is still growing. We would welcome the
researchers to consider this aspect in future research.
6 CONCLUSIONS
3D MULE provide a great potential for engaging
students in innovative, immersive learning
environments. Use of policy considerations for 3D
MULE learning management facilitates the students
and teachers in many ways. As we have
comprehensively shown, students would have
benefited by having a supportive and managed 3D
learning environment, while allowing teachers to
focus more on the educational value of learning
tasks than worrying about the challenges they face
with 3D MULE. Policy based management of 3D
MULE is essential as the situational approaches for
utilizing 3D MUVE for learning would not
otherwise be sustainable. We have presented the two
important aspects identified for policy
considerations through empirical evidence, which
were also validated by previous research. With that
we look forward to facilitating the policy
consideration development through user guidance.
REFERENCES
Allison, C., Miller, A., et al. (2008), Exploratory Learning
for Computer Networking, ICWL, 331-342
Allison C., Miller A., et al., (2010), Educationally
Enhanced Virtual Worlds, 40
th
ASEE/IEEE Frontiers
in Education (FIE), IEEE Press, T4F-1–T4F-6
Allison, C., Miller A., et al., (2011), The Third Dimension
in Open Learning, 41
st
ASEE/IEEE Frontiers in
Education, IEEE Press, T2E-1 – T2E-6
Biggs, J. B. (1996), Enhancing teaching through
constructive alignment, Higher Education, 32(3):347-
364
Costello, A. B. & Osborne, J. W., (2005), Best practices in
exploratory factor analysis: four recommendations for
getting the most from your analysis, Practical
Assessment, Research & Evaluation, 10(7):1-9
Dalgarno, B., et al., (2009), Effectiveness of a Virtual
Laboratory as a preparatory resource for Distance
Education chemistry students, Computers &
Education, 53(3):863-865,
Field A., (2006), Discovering Statistics using SPSS, 2nd
Ed., SAGE, London
Getchell, K., et al. (2010), Games Methodologies and
Immersive Environments for Virtual Fieldwork, IEEE
Transactions on Learning Technologies, 3(4), 281-293
Girvan, C. & Savage, T. (2010), Identifying an appropriate
pedagogy for virtual worlds: A Communal
Constructivism case study, Comp. & Edu., 55,342-349
Kolb, D. A., Boyatzis, R. E. and Mainemelis, C., (2001),
Experiential Learning Theory: Previous Research and
New Directions, J. Sternberg and L. F. Zhang, (Eds):
In Perspectives on Thinking, Learning and Cognitive
Styles, Lawrence Erlbaum: Mahwah, 227
Linden Labs (2003), Second Life, http://www.secondlife.
com
McCaffery J., et al., (2011), Extending the use of virtual
worlds as an educational platform - Network Island:
An advanced learning environment for teaching
Internet routing algorithms, 3rd CSEDU, INSTICC, 1
Perera, I., Allison, C., McCaffery J., and Miller A. (2011),
Towards Effective Blended Learning with 3D MUVE
- An Analysis of Use Case Implementations for 3D
MUVE Learning, 3rd CSEDU2011, 2, 46-55.
CSEDU2012-4thInternationalConferenceonComputerSupportedEducation
198
Perera, I., Allison C., Nicoll, J. R. and T. Sturgeon (2009),
Towards Successful 3D Virtual Learning – A Case
Study on Teaching Human Computer Interaction, In
Proc. of 4th Int. Conf. for Internet Technology and
Secured Transactions ICITST IEEE, 159-164
Pintrich, P. (2000), The role of goal orientation in self-
regulated learning. In M. Boekaerts, P. Pintrich, & M.
Zeidner (Eds.), Handbook of self-regulation, 452–502
Pintrich P. (2003), A motivational science perspective on
the role of student motivation in learning and teaching
contexts. Educational Psychology, 95, 667–686.
Scottish Credit and Qualification Framework (2011), The
SCQF Framework, http://www.scqf.org.uk/
Schunk D. H. (2005), Commentary on self-regulation in
school contexts, Learning & Instruction, 15(2)173-177
Stevens, J. P. (1992), Applied Multivariate Statistics for
the Social Sciences (2nd Ed.) Hillsdale, NJ: Erlbaum
Sturgeon, T., et al. (2009), 802.11 wireless experiments in
a virtual world, SIGCSE Bulletin, ACM, 41(3)85-89
The Open Simulator Project (2007), Open Simulator, http:
//www.opensimulator.org/
Weippl, E. R., (2005), Security in E-Learning, S. Jajodia,
(Eds.): Advances in Information Security, Springer, 16
EFFECTIVEPOLICYBASEDMANAGEMENTOF3DMULE-AnExploratoryStudyTowardsDevelopingStudent
SupportivePolicyConsiderations
199