EFFECTIVE POLICY BASED MANAGEMENT OF 3D MULE

An Exploratory Study Towards Developing Student Supportive Policy

Considerations

Indika Perera, Colin Allison and Alan Miller

School of Computer Science, University of St. Andrews, Fife, KY16 9SX, Scotland, U.K.

Keywords: 3D Multi User Learning Environments, Policy based Management, Second Life, Open Simulator,

Self-regulatory Learning, Learning Environment Management, Student Engagement.

Abstract: Learning environments that are based on 3D Multi User Virtual Environments (3D MUVE) can be referred

as 3D Multi User Learning Environments (3D MULE). 3D MULE used in various educational and research

activities show proven success sufficient to warrant their consideration as a mainstream educational

paradigm. They introduce a platform for diverse learning activities with a novel set of challenges for

teachers and students. Without suitable learning management practices, 3D MULE users can encounter

difficulties during their learning interactions through 3D MUVE functions, although the learning

environment is dynamic and engaging. To overcome this challenge, we researched learner supportive policy

considerations for 3D MULE management. This paper presents an exploratory study with student

engagements (N=32) that identifies key factors, self-regulation and environment management, for policy

considerations. Moreover, the paper critically examines the importance of constructive alignment of

learning activities with the 3D MULE features for useful learning experiences.

1 INTRODUCTION

3D virtual worlds provide unique features for

enhancing technology supported learning through

intuitive activities for learning complex and

advanced concepts. They are particularly appropriate

for educational use due to their alignment with the

Kolb's (Kolb et al., 2001) concept of experiential

learning, and learning through experimentation as a

particular form of exploration (Allison et al., 2008).

For our research we use Second Life (Linden Labs,

2003) and Open Simulator (2007) MUVE; a trend

towards open 3D MUVE such as OpenSim can be

seen recently (Allison et al., 2011), however.

Lack of awareness on 3D MUVE system

functions can cause significant challenges of

learning management for an educationalist, which

can deteriorate the value of 3D MULE and student

motivation for learning. Furthermore, if adequate

management policies are not followed, students may

also find difficulties in adhering to the best practices

when engaged in learning. Managing 3D MULE and

achieving usability and trust in learning can be a

challenge as student interactions may be influenced

by the varying levels of self-regulation practices.

With the interest for extensive use of 3D MULE, we

believe that the use of learner supportive

management policies would positively contribute to

student learning engagement. Therefore, we have

focused on identifying key policy areas for

managing 3D MULE.

Section 2 of this paper describes background

details and our experiences with 3D MULE; section

3 reveals the research methodology and design.

Sections 4 and 5 elaborate the results, analysis,

contributions and study limitations. Section 6

presents future work and conclusions.

2 BACKGROUND AND RELATED

WORK

Interactive 3D virtual environments demonstrate a

great educational potential due to their ability to

engage learners in the exploration, construction and

manipulation of virtual objects, structures and

metaphorical representations of ideas Dalgarno (et

al. 2009). Although the learning experience could be

implemented on other platforms without 3D MUVE

support, the learner experience would be lost, and

193

Perera I., Allison C. and Miller A..

EFFECTIVE POLICY BASED MANAGEMENT OF 3D MULE - An Exploratory Study Towards Developing Student Supportive Policy Considerations .

DOI: 10.5220/0003921101930199

In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 193-199

ISBN: 978-989-8565-06-8

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

users feel contrived (Girvan & Savage, 2010).

Various successful studies for enhancing education

with 3D MULE can be found, recently. However,

most of these studies assume the fact that 3D MUVE

implicitly facilitate learning needs. Since 3D MUVE

are not specifically designed for educational needs,

users have to consider system and user role based

management for successful learning experience

(Perera et al., 2011). Weippl (2005) has considered a

set of factors for e-Learning system management

policy considerations. Previous study on use case

analysis of 3D MULE compared to e-Learning has

identified the significance of environment

management for successful management policy

considerations (Perera et al., 2011).

Technology supported learning environments,

can help to develop specific self-regulatory skills

related to successful engagement in learning

(Dabbagh & Kitsantas, 2004). Pintrich, (2000)

defines self-regulation as “an active, constructive

process whereby learners set goals for their learning

and then attempt to monitor, regulate, and control

their cognition, motivation, and behaviour, guided

and constrained by their goals and the learning

environment contextual features”. Students with

better self-regulatory skills tend to be more

academically motivated and display better learning

(Pintrich, 2003). Schunk (2005) suggests the need of

more research aimed at improving students' self-

regulatory skills as they are engaged in learning and

to examine how learning environment contexts

affect the amount and type of self-regulation

displayed. In this study, we examine self-regulation

and system management for 3D MULE learning.

Several research and educational projects with

3D MUVE are used in the University of St Andrews

as 3D MULE such as, The Laconia Acropolis

Virtual Archaeology (LAVA) (Getchell et al., 2010),

Wireless Island (Sturgeon et al., 2009), Network

Island (McCaffery et al., 2011) and Human

Computer Interaction (HCI) student projects (Perera

et al., 2009). Research on integrating 3D MUVE

with e-Learning infrastructure is conducted (Perera

et al., 2011), which initiated this study.

3 RESEARCH METHODOLOGY

AND EXPERIMENT DESIGN

Student behaviour and system administration have

been widely considered aspects for effective

learning environment design. Based on our previous

studies on 3D MUVE use cases and role analysis

(Perera et al., 2011) and the 3D MUVE function

behaviours, we hypothesised these two parameters to

be the most influential factors for successful policy

considerations for 3D MULE management.

Furthermore, we decided to extend the study to

investigate the impact of these two factors on

student engagement with the 3D MULE. Therefore,

we examine three factors: student behaviour, system

(environment) management and student engagement

with the environment as the research variables.

Importantly, the engagement with 3D MULE may

not necessarily represent the student engagement

with the learning, although there can be a positive

correlation if the learning tasks are constructively

aligned (Biggs, 1996). However, the opposite; i.e., if

the student engagement with the 3D MULE is low,

so it is with the learning tasks that depend on 3D

MULE, obviously. There are unique advantages of

using 3D MUVE for teaching and learner support,

which we may not achieve from the other methods.

Deductively, if students do not engage with 3D

MULE, there is a high tendency of them having less

engagement with their learning as well. Hence, our

policy considerations for 3D MULE management

should not negatively affect the student engagement.

The following research hypotheses were defined

to examine the supposed variables and their impact.

H1: Student behaviour with self-regulation is a

major factor of the successful 3D MULE

management

H2: System environment management is a major

factor of the success of 3D MULE management

H3: Student self-regulatory behaviour has a

positive and significant effect on student

engagement with 3D MULE

H4: System environment management has a

positive and significant effect on student

engagement with 3D MULE

3.1 Experiment Design

We followed a two phase study; first, students from

two course modules experienced the 3D MUVE

supported learning environment (in OpenSim) as a

part of their studies. In contrast to Second Life,

OpenSim gives several advantages (Allison et al,

2010). It is important that students participate in a

credit bearing learning session designed with 3D

MUVE for the data accuracy than following an

artificial task. Further, it helps students to

consciously associate the experience they had. The

two modules have different learning objectives

levels in Scottish Credits and Qualification

Framework (SCQF, 2007). This study focused on

student engagement with the environment; although

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

194

students had different levels of the same learning

task, both samples were similar with respect to our

measures. The differences in the modules did not

affect the experiment; hence can be considered as a

single sample of 59 students for the data analysis.

Table 1: Considered module information.

Module Information SCQF Level

CS3102 Data communication and Networks

- 31 Students

10- Hons.

Degree

CS5021 Advanced Networks and Distributed

Systems - 28 Students

11-PG

Degree(MSc)

The learning environment, Wireless Island

(Sturgeon et al., 2009), is a dedicated region for

facilitating learning and teaching wireless

communication through interactive simulations with

configuration settings and supplementary learning

content for exploration. To facilitate small group

learning with a less competitive environment

interaction, we set up 5 identical Wireless Islands

(256x256 m

virtual space; 6 students per island) in

the OpenSim environment. Students were assigned

to regions as their home place to start their learning.

For the second phase, a questionnaire with 15

questions, divided into two sections: avatar

behaviour–7 questions, and 3D MULE management

–8 questions, was used. The 8 questions in the 3D

MULE management section had some relevance on

the two factors that we are investigating, self-

regulation and environment management; however,

the questions did not directly portray the variables.

We decided to confirm through the statistical

analysis, therefore, included as 8 related questions.

4 RESULTS AND ANALYSIS

4.1 Observed Student Engagements

Avatar appearance change can be a fun, although

creating attractive appearances within a limited time

can be challenging. OpenSim with Hippo viewer

gave an additional step on changing user appearance

compared to Second Life. Users first have to create a

body-part, edit it and then wear it to change the

shape of the default avatar. However, few students

spent long enough to create more sophisticated

shapes, clothes and even change the avatar gender

(default avatar shape is shown in the top-left image

of Fig. 1). Postgraduate students were less keen on

changing their avatar, whereas many undergraduates

went to a further step by comparing the avatar

appearances with their friends’. However, the

student commitment for making their avatars look

good should not be underestimated as it can take

substantial portions of their learning time.

Content creation is one of the fundamental

interactivity mechanisms available in 3D MUVE.

Allowing students to create attractive 3D shapes

makes them engaged with the environment

passionately. As observed, students tried a range of

constructions and editing of the existing objects.

Figure 1: Observed student interactions.

Some of the alterations directly affected the

learning experience; activities such as wearing the

control buttons of the media displays, moving and

changing the internal arrangement of the lecture

theatre, and creating constructs on the simulation

area (Fig. 1), introduced some disturbances. These

can be discouraged through usable policy

considerations. A student interaction caused the

learning environment to be significantly altered

compared to its original layout (Fig.1 last row). This

observation was a one-off incident, as the majority

of students refrained from changing land settings.

Compared to the postgraduate (Masters) students,

the undergraduate (Honours) students showed high

interactivity, resulted in a range of user-created

objects, altered content and changed land terrain.

The undergraduates were keen on exploring game-

like features, and associate their friends for

collaborative activities, although those activities

were not related with the learning. Students that

were keen on completing their tasks may have had

less motivation to explore the 3D MUVE, however.

Students were allowed to follow their preferred

behaviour and environment interactions as a mean of

learning through exploration (Kolb et al., 2001)

without any restriction. In this exploratory study, we

wanted to examine these actions as empirical

EFFECTIVEPOLICYBASEDMANAGEMENTOF3DMULE-AnExploratoryStudyTowardsDevelopingStudent

SupportivePolicyConsiderations

195

evidence; an assurance was given that their

behaviour does not affect their grades, but the

completion of the learning tasks.

4.2 Analysis of Questionnaire Data

We received 32 completed questionnaires (54.28%),

[20 (71.4%) from postgraduate and 12 (38.7%) from

undergraduate students]. An initial observation of

the question characteristics and the descriptive

statistics helped to understand the user responds and

possible classifications. Analysis of the question and

responds resulted in preliminary clustering of the

questions into two major categories: (Q1-Q7) user

engagement and (Q8-Q15) 3D MULE management.

The questions in user engagement section, based

on different student behavioural activities in the

environment, indicated a higher internal consistency

(Cronbach’s α = 0.802). The internal consistency

validated the combined use of questions to represent

the associated variable. The 3D MULE management

section was designed to examine the two prime

variables associating the research hypotheses. We

have statistically analysed for accurate variable

identification as a method to test the hypotheses.

Exploratory Factor Analysis (EFA) used to test

the Q8-Q15 question set. Pre-tests were conducted

for the fitness of data to be used for EFA.

Bartlett

Test of Sphericity, a strict test on sampling and

suitable correlations, was performed using PASW

(18.0), and obtained χ

=155.257, p<.001, suggesting

that the correlation matrix (R-Matrix) items can be

clustered based on relationships; the null hypothesis

of R-Matrix being an Identity Matrix can be rejected

with significance indicating a higher level of fitness

for EFA.

Additionally, the Kaiser-Meyer-Olkin

(KMO) test to examine the accuracy of using the

data sample for the EFA; KMO value =0.714(>0.7;

Field, 2006) validated the fitness. The sample size to

variable, N:p was higher and meets the guidelines

given by Costello & Osborne (2005).

We used EFA to test H1 and H2. Principal

Component Analysis (PCA) was used to extract

factors; two factors (Eigenvalues>1.0) was obtained.

These highest two factors contribute nearly 72% of

the total variation of the aspect 3D MULE

Management.

Further, we employed Orthogonal

Verimax Rotation with Kaizer Normalization and

obtained the rotated factor loadings (Fig. 2).

To remove weak loadings, we followed Stevens

(1992) and Field (2006) suggestions, and used .6 as

the cut-off, considering the exploratory nature of the

analysis. The Orthogonal Verimax Rotation seems

accurate as the two factors relatively equally

contribute (37.13% and 34.74%) to the underlying

aspect. The Component Transformation Matrix

showed symmetry over the diagonal, indicating the

Orthogonal Rotation is accurate, and the rotated

factor loadings are correct. As the rotated factor

loadings indicated, questions Q9, Q15, Q10 & Q8

were considered as a one variable. The objectives of

the questions suggested a common parameter. We

concluded it as the student behaviour with Self-

regulation, as expected. The second factor represents

Q11, Q14, Q12 & Q13 questions and strongly

relates the student preference and impact on the

system control, administration and management of

Table 2: Questionnaire items and the descriptive statistics on item scores.

No Question Mean Mode Std. Dev. Std. Error

Avatar Engagement

Q1 I changed my appearance as I like to appear 3.09 3 0.466 0.082

Q2 I created content objects in the environment 3.69 4 0.471 0.083

Q3 I tried to change the land or content objects in the learning environment 3.66 4 0.483 0.085

Q4 I communicated with others regularly

3.53 4 0.507 0.090

Q5 I have followed other avatars collaboratively during my learning 3.44 3 0.504 0.089

Q6 I moved to all the places in my island and teleported to other islands 3.81 4 0.397 0.070

Q7 My activities in the environment resulted in a high engagement with my learning

tasks

3.97 4 0.40 0.071

Self-Regulation

Q8 I think my behaviour affected others’ learning 3.31 3 0.592 0.104

Q9 The open space and other avatars made me to interact as in a real-world learning

session

4.05 4 0.354 0.064

Q10 Use of real identities increases the proper behaviour of students 4.02 4 0.309 0.056

Q15 Students should responsibly use the learning environment 4.10 4 0.296 0.051

Environment Management

Q11 Land and Content management controls are important to manage environment 3.78 4 0.420 0.074

Q12 System control and management practices are important for a reliable learning 3.91 4 0.296 0.052

Q13 System management settings should not reduce the 3D MUVE usability 4.19 4 0.592 0.105

Q14 Appropriate system security & controls ensure a successful learning experience 3.89 4 0.390 0.070

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

196



Figure 2: Rotated Component Matrix and Component Plot

in Rotated Space.

the 3D MUVE. Therefore, we defined this variable

as Environment Management to cover all the aspects

that associate. The internal consistencies among the

question items within the three variables found to be

very high (Cronbach α >.8). The items on self-

regulation and environment management also meet

the requirements to be considered collectively to

represent the variables. One-Sample Kolmogorov-

Smirnov test for normality on the variables student

engagement, self-regulation, environment

management resulted in [X~N(3.598,0.283),

α=0.997], [X~N(3.876,0.378), α=0.516],

[X~N(3.95,0.175), α=0.796], respectively, retaining

the hypothesis of normal distribution confirming the

collective use of items with normality.

Table 3: Regression model summary.

Variable β Std. Err. t-value Sig.

Self-Regulation .240 .097 2.482 0.019*

Environment

Management

.657 .092 7.312 0.000**

* p < 0.05, ** p < 0.001

The Spearman rho between Self-Regulation and

Environment Management is 0.398 indicating a

weak positive relationship (p<0.05). This is

important since the two variables measure

sufficiently different parameters and the inter-

relationship is insignificant. Therefore, we conclude

that, Self-Regulation and Environment Management

are sufficiently independent as measuring two

different aspects, which further proves the EFA and

selection of variables to represent the 3D MULE

Management aspect.

Moreover, there aren’t any

other strong and significant variable revealed

through the EFA. Therefore, the research hypothesis

H1 and H2 are substantiated. To test the research

hypotheses H3 and H4, we used regression analysis.

A sample size test was done for the fitness for

regression analysis. As in Field (2006), for the test

statistics of anticipated large effect (F

=0.35),

Number of predictors (n=2), Probability level of

Significance (α=0.05) with the desired statistical

Power level of (1-β=0.8), the minimum required

sample size was 31. Therefore, our sample size

N=32 (>31) suites well for the analysis and the

regression model is reliable.

Linear regression analysis, R

=0.759, indicates

that about 75.9% of the variation in the student

engagement is determined by the environment

management and student self-regulation in 3D

MULE, as a combined effect. ANOVA of the model

fit showed, that the regression model significantly

explains the Student Engagement from the variables

Environment Management and Self-Regulation

(p<0.001). As the variable relationship with

predictor parameters of the model shown in table 3,

the path coefficients are .240 for Self-Regulation,

which is significant (p<0.05) and .657 for

Environment Management with significance

(p<.001). Therefore, the research model

substantiates our hypotheses H3 & H4.

5 DISCUSSION

Available space prevents a more comprehensive

discussion on the individual questions and the

recorded scores; the question responses are self-

explanatory with the results shown in Table2.

However, briefly, Q7 suggests an important

relationship between the environment engagement

and learning engagement, which is highly

recommended for further study. The Q5 and Q8

show the importance of associating collaborative

learning tasks with the available 3D MUVE

facilities. Student collaboration occurs through

learner interaction while interacting with the 3D

MUVE that provides supports rather than barriers to

learning (Girvan & Savage, 2010).

EFA and consistency tests resulted in the

identification of the two variables: self-regulation

and system environment management as the major

factors of 3D MULE management, confirming the

hypotheses H1 and H2. Therefore, we suggest these

two parameters as main consideration areas for 3D

MULE management policy development. With

reference to H3 and H4, the student self-regulation

on learning activities and 3D MUVE management

practices result in a significant positive effect on the

student engagement with the environment and

learning tasks. We conclude that, for constructive

EFFECTIVEPOLICYBASEDMANAGEMENTOF3DMULE-AnExploratoryStudyTowardsDevelopingStudent

SupportivePolicyConsiderations

197

and successful 3D MULE student engagement, we

must identify and implement policies for student

self-regulation and 3D MUVE system management.

Thus, we validate our research hypotheses on

considering self-regulation and environment

management of 3D MUVE as a prime factor for 3D

MULE management policy considerations.

The environment management indicated the

highest positive impact on increasing the student

engagement. Although, students entertain

themselves by various environment engagements

they also felt the difficulty of task coordination and

unsupportive avatar behaviour during learning

engagements. For example, lecture displays reset

whenever an avatar hits the play button, disturbing

the other viewers. Also, if a student’s simulation

arrangement is too close to another’s setup,

simulation interferences were observed. These can

be easily solved if we implement suitable

environment management policy considerations.

Considering the student opinions and our

observations, we suggest the importance of cohesive

learning activities with 3D MULE user engagement

aspects through constructive alignment (Biggs,

1996). This would enable students to spend their

time highly engaged with the 3D MULE, while

achieving their learning goals comfortably.

Careful analyses and tests were employed to

minimise the impact of the following limitations.

Due to the nature of the research, the study sample

was limited to a particular set of students. These

students have provided their feedback and answers

based on their experiences, which we validated

through observation. Therefore, we consider the data

received as accurate and conclusive though the study

had a relatively small sample, due to resource

constraints. The questions used were appropriately

designed, although they have yet to be examined for

psychometric measures. It is a challenge to find a

widely accepted standard set of psychometric

measures in particular for 3D MULE, as the field of

study is still growing. We would welcome the

researchers to consider this aspect in future research.

6 CONCLUSIONS

3D MULE provide a great potential for engaging

students in innovative, immersive learning

environments. Use of policy considerations for 3D

MULE learning management facilitates the students

and teachers in many ways. As we have

comprehensively shown, students would have

benefited by having a supportive and managed 3D

learning environment, while allowing teachers to

focus more on the educational value of learning

tasks than worrying about the challenges they face

with 3D MULE. Policy based management of 3D

MULE is essential as the situational approaches for

utilizing 3D MUVE for learning would not

otherwise be sustainable. We have presented the two

important aspects identified for policy

considerations through empirical evidence, which

were also validated by previous research. With that

we look forward to facilitating the policy

consideration development through user guidance.

REFERENCES

Allison, C., Miller, A., et al. (2008), Exploratory Learning

for Computer Networking, ICWL, 331-342

Allison C., Miller A., et al., (2010), Educationally

Enhanced Virtual Worlds, 40

ASEE/IEEE Frontiers

in Education (FIE), IEEE Press, T4F-1–T4F-6

Allison, C., Miller A., et al., (2011), The Third Dimension

in Open Learning, 41

ASEE/IEEE Frontiers in

Education, IEEE Press, T2E-1 – T2E-6

Biggs, J. B. (1996), Enhancing teaching through

constructive alignment, Higher Education, 32(3):347-

364

Costello, A. B. & Osborne, J. W., (2005), Best practices in

exploratory factor analysis: four recommendations for

getting the most from your analysis, Practical

Assessment, Research & Evaluation, 10(7):1-9

Dalgarno, B., et al., (2009), Effectiveness of a Virtual

Laboratory as a preparatory resource for Distance

Education chemistry students, Computers &

Education, 53(3):863-865,

Field A., (2006), Discovering Statistics using SPSS, 2nd

Ed., SAGE, London

Getchell, K., et al. (2010), Games Methodologies and

Immersive Environments for Virtual Fieldwork, IEEE

Transactions on Learning Technologies, 3(4), 281-293

Girvan, C. & Savage, T. (2010), Identifying an appropriate

pedagogy for virtual worlds: A Communal

Constructivism case study, Comp. & Edu., 55,342-349

Kolb, D. A., Boyatzis, R. E. and Mainemelis, C., (2001),

Experiential Learning Theory: Previous Research and

New Directions, J. Sternberg and L. F. Zhang, (Eds):

In Perspectives on Thinking, Learning and Cognitive

Styles, Lawrence Erlbaum: Mahwah, 227

Linden Labs (2003), Second Life, http://www.secondlife.

com

McCaffery J., et al., (2011), Extending the use of virtual

worlds as an educational platform - Network Island:

An advanced learning environment for teaching

Internet routing algorithms, 3rd CSEDU, INSTICC, 1

Perera, I., Allison, C., McCaffery J., and Miller A. (2011),

Towards Effective Blended Learning with 3D MUVE

- An Analysis of Use Case Implementations for 3D

MUVE Learning, 3rd CSEDU2011, 2, 46-55.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

198

Perera, I., Allison C., Nicoll, J. R. and T. Sturgeon (2009),

Towards Successful 3D Virtual Learning – A Case

Study on Teaching Human Computer Interaction, In

Proc. of 4th Int. Conf. for Internet Technology and

Secured Transactions ICITST IEEE, 159-164

Pintrich, P. (2000), The role of goal orientation in self-

regulated learning. In M. Boekaerts, P. Pintrich, & M.

Zeidner (Eds.), Handbook of self-regulation, 452–502

Pintrich P. (2003), A motivational science perspective on

the role of student motivation in learning and teaching

contexts. Educational Psychology, 95, 667–686.

Scottish Credit and Qualification Framework (2011), The

SCQF Framework, http://www.scqf.org.uk/

Schunk D. H. (2005), Commentary on self-regulation in

school contexts, Learning & Instruction, 15(2)173-177

Stevens, J. P. (1992), Applied Multivariate Statistics for

the Social Sciences (2nd Ed.) Hillsdale, NJ: Erlbaum

Sturgeon, T., et al. (2009), 802.11 wireless experiments in

a virtual world, SIGCSE Bulletin, ACM, 41(3)85-89

The Open Simulator Project (2007), Open Simulator, http:

//www.opensimulator.org/

Weippl, E. R., (2005), Security in E-Learning, S. Jajodia,

(Eds.): Advances in Information Security, Springer, 16

EFFECTIVEPOLICYBASEDMANAGEMENTOF3DMULE-AnExploratoryStudyTowardsDevelopingStudent

SupportivePolicyConsiderations

199