Water Cycle in Nature – An Innovative Virtual Reality and Virtual

Lab: Improving Learning Experience of Primary School Students

Diana Bogusevschi and Gabriel-Miro Muntean

Dublin City University, Ireland

Keywords: Virtual Reality, Virtual Laboratory, Computer-based Learning, TEL, Learner Experience, STEM, Primary

School.

Abstract: A technology-enhanced learning (TEL) application, “Water Cycle in Nature”, that focuses on the physics

phenomena part of the natural water cycle and precipitation formation was employed in a small-scale

educational pilot carried out in a primary school in Ireland, as part of the European Horizon 2020 NEWTON

project. This application contains 3D immersive computer-based virtual reality and experimental laboratory

simulations. 58 primary school children took part in this pilot, split in two groups, one control and one

experimental, with 29 students in each. The goal of the study presented in this paper was to assess the benefits

of the Water Cycle in Nature application both in learner experience and usability and knowledge gain. The

results show good outcomes in usability and learner experience. In terms of knowledge gain it has been shown

that the excitement of the experimental group students towards the game might have created a barrier in terms

of learning improvement and the NEWTON application will serve better as a revision tool.

1 INTRODUCTION

Subjects in science, technology and maths (STEM)

are currently suffering an increased lack of interest

from students starting from primary to third level

institutions. It is very important to capture learners’

attention to STEM subjects early on, starting in

primary school, and encouraging them to pursue these

subjects in future education. Numerous studies have

been performed on this, such as investigating content

and language integrated learning in OpenSimulator

Project (CLILiOP), focusing on Geography by

employing Virtual Reality (VR) (Fokides and

Zamplouli, July 2017), showing better cognitive

results for the experimental group of learners and a

higher employment of geographical terms in

knowledge post-tests, compared to the control class.

VR is also employed in the field of primary school

mathematics, investigating the benefits of

OpenSimulator VR environment combined with

game-based learning, showing significant

improvement when using the combination between

the two techniques (Kim and Ke, 2017).

Another TEL technique employed in STEM

subjects is Augmented Reality (AR) showing much

potential in attracting learners of all ages and levels

to science-related education, such as in (Cuendet, et

al., 2013), where three different AR systems,

TinkerLamp, Tapcarp and Kaleidoscope, were

employed for geometry teaching in a primary school

setting, showing great usability and integration into

the classroom. In (Sommerauer and Muller, 2014) AR

was employed in an informal environment at a

mathematics exhibition, where participants of various

ages took part, including primary level students,

showing that all AR-enhanced exhibits performed

significantly better in terms of knowledge acquisition

and retention compared to non-AR exhibits. The

benefits of combining AR with game-based learning

are investigated in (Pellas, et al., 2018) presenting

multiple benefits on both knowledge gain and

learning motivation, specifically in STEM education.

The concept of game and technology-based primary

level maths teaching is also investigated in (Misfeldt

and Zacho, 2016), where its benefits are observed

from the point of view of both teachers and learners.

Modern education is trying to stay in line with

technology, such as iPads being employed in many

schools for all subjects, including maths. For

example, in (Hilton, 2018), where its effect on

learners’ attitude and motivation toward maths is

investigated, showing a positive influence from both

angles. The use of tablets in education is also

304

Bogusevschi, D. and Muntean, G.

Water Cycle in Nature – An Innovative Virtual Reality and Virtual Lab: Improving Learning Experience of Primary School Students.

DOI: 10.5220/0007760803040309

In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 304-309

ISBN: 978-989-758-367-4

investigated in (Fokides and Atsikpasi, 2017),

specifically for teaching plants to primary level

students, showing a positive impact on knowledge

acquisition and improvement in both collaborative

and independent learning for the experimental group.

Mobile devices have also been combined with

collaborative learning in (Iglesias Rodrguez, et al.,

2017) presenting major benefits for primary school

learners. The use of tablets in primary and secondary

education, including for STEM subjects, has been

investigated in (Haßler, et al., 2016), where most of

the examined case studies showed positive learning

outcomes. Such primary education benefits are also

shown in (Domingo and Gargante, 2016), improving

access to content and increasing engagement from

pupils.

This paper describes the use of the Horizon 2020

NEWTON project computer-based application,

Water Cycle in Nature, in a primary school in Dublin,

Ireland, focusing on physics, specifically on

precipitation formation. A research study on 5

primary school classes was carried out, one control

and one experimental, examining the benefits of the

Water Cycle in Nature application for knowledge

gain, learner experience and usability. The next two

sub-sections give a brief description of the Horizon

2020 NEWTON project and the Water Cycle in

Nature application. The case study is described in

Section 2, followed by the obtained results in Section

3. The summary of the paper and its conclusions are

presented in Section 4.

1.1 European Horizon 2020 NEWTON

Project

Horizon 2020 NEWTON project aims to design,

develop and deploy innovative solutions for TEL

including innovative pedagogies such as adaptive

multimedia and multi-sensorial content delivery

mechanisms (Zou, et al., 2018), (Bi, et al., 2018),

personalisation and gamification solutions (Lynch

and Ghergulescu, July 2017), Virtual Labs (VL) and

fabrication labs (Togou, et al., July 2018), problem-

based, game-oriented, and flipped-classroom-based

learning (El Mawas, et al., 2018). All NEWTON

project solutions are employed using the NEWTON

project technology enhanced learning platform,

NEWTELP, to be used by teachers for course creation

and both knowledge and qualitative assessment, and

by students with the primary focus on learning course

material and completing knowledge tests and

questionnaires (Montandon, et al., June 2018).

1.2 Water Cycle in Nature Application

The Water Cycle in Nature application was developed

by NEWTON project consortium partner SIVECO in

Romania and is one of many applications employed

in small and large-scale technology enhanced

educational pilots with the objective of assessing

learner satisfaction and knowledge gain. The Water

Cycle in Nature application is a VL combined with

VR technology, where the content is explored by

students though immersive multimedia 3D in two

separate settings: Nature Environment for presenting

and Experimental VL environment (Bogusevschi, et

al., 2018), (Bogusevschi, et al., April 2018) for

reinforcing the previously presented definitions, such

as vaporisation, evaporation, boiling and

condensation.

2 CASE STUDY DESCRIPTION

2.1 Evaluation Methodology

The Water Cycle in Nature application was employed

in a primary school in Dublin, Ireland, St. Patrick’s

Boys’ National School (BNS), where two 5

classes,

with 29 pupils in each class aging from 10 to 11 years

old, participated and randomly assigned as control

and experimental group. Ethics approval was

obtained from the DCU Ethics Committee and this

evaluation meets all ethics requirements.

The experimental class took part in the NEWTON

approach lesson, where the Water Cycle in Nature

application was employed. The control class

participated in a classic approach lesson, developed

by the NEWTON Project team, ensuring that the

educational content in both lessons, classic approach

and NEWTON approach, were identical, and was

provided by their usual teacher. Pre-tests were carried

out in both classes before their respective lesson,

assessing the participating pupils’ knowledge level of

the topic. Following each lesson, post-tests were

provided to students, investigating knowledge gain.

Following all the compulsory steps of the small-scale

study, pre-test, classic approach lesson, post-test, the

control group also interacted with the Water Cycle in

Nature application, allowing the NEWTON project

researchers to assess the comparison in terms of

learner satisfaction between the classic approach and

the NEWTON approach. Following the interaction

with the application, both classes completed a

Learner Satisfaction Questionnaire (LSQ), containing

the following questions: Q1- The video game and the

experiments that I did in the lab from the video (this

Water Cycle in Nature – An Innovative Virtual Reality and Virtual Lab: Improving Learning Experience of Primary School Students

305

is called a virtual lab!) helped me to better understand

vaporisation and condensation processes; Q2- The

video game and the experiments that I did in the

virtual lab helped me to learn easier about the

vaporisation and condensation processes; Q3-I

enjoyed this lesson that included the video game and

the experiments in the virtual lab; Q4-The

experiments that I did in the virtual lab made the

lesson more practical; Q5-The video game distracted

me from learning; Q6-I would like to have more

lessons that include video games and doing

experiments in virtual labs; Q7-

Comments/Suggestions (Bogusevschi, et al., April

2018).

2.2 Data Collection

For the experimental group, the Water Cycle in

Nature application was uploaded on NEWTON

Project lap-tops, as the school PCs did not have the

necessary specifications to support the application.

After the post-test knowledge assessment was

performed, the control class also interacted with the

application.

The DCU NEWTON Project researchers

supervised the experimental approach, providing

support and helping students when necessary,

collecting all the paper-based knowledge tests and

LSQs. The classic approach lesson was carried out

simultaneously with the experimental approach

lesson, by the usual control class teacher. Student IDs

were employed for both groups, in order to ensure

anonymization.

The knowledge pre-test and post-test contained seven

and eight questions respectively, each with a

maximum of 10 points (Bogusevschi, et al., 2018).

3 RESULTS

3.1 Learner Experience

The learner experience was assessed based on LSQ

questions Q1 to Q6. The LSQ percentage results for

the experimental and control classes are presented in

Figure 1 and Figure 2. A 5-Likert scale was employed

during the LSQ, with the following answers Strongly

Agree (SA), Agree (A), Neutral (N), Disagree (D) and

Strongly Disagree (SD). One control group student

did not provide answers to questions Q4, Q5 and Q6.

Figure 1: Experimental Group Learners’ LSQ results.

Figure 2: Control Group Learners’ LSQ results.

It can be observed that a much higher percentage

of control group students agreed that the Water Cycle

in Nature application VL helped them better

understand vaporisation and condensation processes

(Q1), at just under 83%, compared to ~52% for the

experimental class. This might point to the fact that

the NEWTON approach worked better as a revision

tool for the control class, as the learners from this

group were firstly presented the relevant physics

topics during a classic approach lesson. A higher

percentage of control group students considered that

the video game and the experiments helped them to

learn easier about the vaporisation and condensation

processes (Q2), with over 86% compared to just

under 69% of the experimental group students, which

also points to a higher efficiency of the application as

a revision tool, rather than an introduction tool. A

similar percentage of learners in both classes enjoyed

the application (Q3), with ~93% in the control class

and ~86% in the experimental class. The same

percentage of students (75.86%) in both classes

thought that the Water Cycle in Nature application

virtual experiments made the lesson more practical

(Q4). Just over 24% of learners in each class thought

that the application was distracting them from

learning (Q5) and the vast majority of participating

students would like to have more lessons that include

video games and doing experiments in virtual labs

CSEDU 2019 - 11th International Conference on Computer Supported Education

306

(Q6), at 93% of experimental class and just over 96%

of the control class learners.

3.2 Application Usability

The usability of the Water Cycle in Nature

application was assessed using the provided LSQ Q7

answers. In each class, 11 students (37.9%) chose not

to provide an answer to this question. The majority of

the remaining 62.1% provided positive comments,

such as “Good lesson”, “I enjoyed it and I would do

it again”, “Very fun and help me learn about

vaporization and condensation”, “Very fun and easy

to learn definitely want to learn more things like this

as normal school is boring” in the control class and

“It was Awesome”, “It was really fun it helped me to

learn about vaporization and condensation”, “It was

amazing and the best I love learning that way”, “I

think was so cool thank you so much” in the

experimental class.

It appeared that the control class was somewhat

more positive in their comments, as they used the

application to review what was already presented to

them in the classic approach lesson and they were

using the NEWTON approach lesson more so as a

revision tool. Some of the experimental class students

perceived parts of the Water Cycle in Nature

application slightly boring, expressing the hope to

have a more gamified experience and to have more

freedom in the VL, in terms of experiments to carry

out. Some students provided commented on the audio

track, suggesting having a more engaging voice-over.

3.3 Knowledge Acquisition Assessment

The learning impact assessment was investigated

using the pre-tests provided to both classes prior the

classic approach for control class and NEWTON

approach for the experimental class and the post-tests

employed after each lesson. The average grades for

the pre-test and post-test are presented in Figure 3. It

is seen that the experimental class had a slightly

higher average pre-test grade. A t-test between the

two pre-tests showed that the experimental group’s

higher average pre-test mark was of no statistical

significance, at α = 0.05 (t(56) = 1.7423, p = 0.087).

Following the classic and NEWTON approach

lessons the average grades improved for both classes.

The experimental class exhibited a 15.21%

improvement, with 55% of students providing

improved grades, whereas the control group had a

much higher knowledge gain, at over 85%, with 82%

of students having improved grades. The

experimental group shows improvement of no

statistical significance, at α = 0.05 (t(28) = 1.243, p =

0.2239). The grades improvement for the control

group is statistically significant, at α = 0.05 (t(28) =

5.0517, p = 0.0001).

Figure 3: Average Pre and Post-test Mean grades for the

experimental and control groups.

4 CONCLUSIONS

The primary school small-scale TEL pilot described

in this paper investigates the usability and learner

experience, as well as knowledge gain assessment of

the Horizon 2020 NEWTON Project Water Cycle in

Nature application. A small-scale educational pilot

was conducted in St. Patrick’s BNS in Dublin,

Ireland. Two classes of 29 students in each took part

in the case study, one class exposed to the NEWTON

project application as the experimental group and the

other class, where the classic teacher-based approach

was used, as the control group. Both participating

groups where provided knowledge tests before and

after the lessons, teacher-based or computer-based, in

order to assess the learning outcomes for each

teaching approach. Following the post-tests after the

classic approach lesson, the control class was also

exposed to the application. Both classes completed a

LSQ, investigating Learner Experience and

Application Usability. In terms of knowledge gain,

both classes showed improvement, however it was

observed that the control class performed much better

compared to the experimental class. The experimental

class appeared much more excited about the Water

Cycle in Nature application which might have created

a barrier to achieving significant knowledge gain, as

they were more interested in the visual aspects of the

application, rather than its educational content. The

classic approach presentation was provided to the

control group by their usual teacher, who allowed

additional questions from students during the classic

approach, which created an advantage for the control

class. In the NEWTON approach lesson, the

experimental class did not ask any questions, as they

Water Cycle in Nature – An Innovative Virtual Reality and Virtual Lab: Improving Learning Experience of Primary School Students

307

were very focused on the application’s immersive

multimedia 3D simulation and not on the actual

educational content. It also has to be noted that during

the experimental approach, the usual teacher was not

present, which might have introduced a very high

sense of freedom for the experimental group learners,

enabling them to concentrate only on the game, which

was a very new school experience for them, rather

than on learning. This case study reinforced the

perception that teachers’ leadership is extremely

valuable during TEL lessons (Bogusevschi, et al.,

n.d.).

Very good results were observed in terms of

Learners Experience and Application Usability. The

control class reported the application much more

useful when learning about precipitation formation,

compared to the experimental group. This might be

due to the fact that, having been presented the topic

in a classic approach manner first, the control group

considered the NEWTON Project Water Cycle in

Nature application as a revision tool. Following the

LSQ comments provided by the two groups, the

application was updated and localised, and it will be

employed as part of a large-scale pilot in various

European countries (Ireland, Slovakia and Romania)

as part of the Earth Course (Bogusevschi, et al., 2018)

that will be provided to students using multiple

NEWTON project technologies, which, following the

findings in the small-scale pilots presented in this

paper, will also assess the NEWTON approach as

both an introductory and as a revision tool.

ACKNOWLEDGEMENTS

This research is supported by the NEWTON project

(http://www.newtonproject.eu/) funded under the

European Union’s Horizon 2020 Research and

Innovation programme, Grant Agreement no.

688503.

REFERENCES

Bi, T. et al., 2018. A DASH-based Mulsemedia Adaptive

Delivery Solution. Amsterdam, The Netherlands, s.n.

Bogusevschi, D., Bratu, M., Ghergulescu, I. and Muntean,

C. H. a. M. G.-M., April 2018. Primary School STEM

Education: Using 3D Computer-based Virtual Reality

and Experimental Laboratory Simulation in a Physics

Case Study. Dublin, Ireland, s.n.

Bogusevschi, D., Maddi, M. and Muntean, G.-M., n.d.

Teachers’ Impact and Feedback Related to Technology

Enhanced Learning in STEM Education in Primary and

Secondary Schools. Dublin, Ireland, s.n.

Bogusevschi, D., Muntean, C.-H., Gorgi, N. and Muntean,

G.-M., 2018. EARTH COURSE: PRIMARY SCHOOL

LARGE-SCALE PILOT ON STEM EDUCATION.

Palma de Mallorca,Spain, s.n.

Bogusevschi, D. et al., 2018. Water Cycle in Nature: Small-

Scale STEM Education Pilot. Amsterdam, The

Netherlands, s.n.

Cuendet, S., Bonnard, Q., Do-Lenh, S. and Dillenbourg, P.,

2013. Designing augmented reality for the classroom.

Computers and Education, October, Volume 68, pp.

557-569.

Domingo, M. G. and Gargante, A. B., 2016. Exploring the

use of educational technology in primary education:

Teachers' perception of mobile technology learning

impacts and applications' use in the classroom.

Computers in Human Behavior, Volume 56, pp. 21-28.

El Mawas, N. et al., 2018. Final Frontier Game: A Case

Study on Learner Experience. Madeira, Portugal, s.n.

Fokides, E. and Atsikpasi, P., 2017. Tablets in education.

Results for the initiative ETiE for teaching plants to

primary school studnets. Educ Inf Technol, Volume 22,

pp. 2545-2563.

Fokides, E. and Zamplouli, C., July 2017. Content and

language integrated learning in OpenSimulator project.

results of a pilot implementation in Greece. education

and Information Technologies, 22(4), pp. 1479-1496.

Haßler, B., Major, L. and Hennessy, S., 2016. Tablet use in

schools: a critical review of the evidence for learning

outcomes. Journal of Computer Assisted Learning,

Volume 32, pp. 139-156.

Hilton, A., 2018. Engaging Primary School Students in

Mathematics: Can iPads Make a Difference?. Int J of

Sci and Math Educ, Volume 16, pp. 145-165.

Iglesias Rodrguez, A., Garca Riaza, B. and Sanchez

Gomez, M. C., 2017. Collaborative learning and mobile

devices: An educational experience in Primary

Education. Computers in Human Behavior, Volume 72,

pp. 664-677.

Kim, H. and Ke, F., 2017. Effects of game-based learning

in an OpenSim-supported virtual environment on

mathematical performance. Interactive Learning

environments, 25(4), pp. 543-557.

Lynch, T. and Ghergulescu, I., July 2017. Large Scale

Evaluation of Learning Flow. Timisoara, Romania, s.n.

Misfeldt, M. and Zacho, L., 2016. Supporting primary-level

mathematics teachers' collaboration in designing and

using technology-based scenarios. J Math Teacher

Educ, Volume 19, pp. 227-241.

Montandon, L. et al., June 2018. Multi-dimensional

Approach for the Pedagogical Assessment in STEM

Technology Enhanced Learning. Amsterdam, The

Netherlands, s.n.

Pellas, N., Fotaris, P., Kazanidis, I. & Wells, D., 2018.

Augmenting the learning experience in primary and

secondary education: a systematic review of recent

trends in augmented reality game-based learning.

Virtual Reality.

CSEDU 2019 - 11th International Conference on Computer Supported Education

308

Sommerauer, P. and Muller, O., 2014. Augmented Reality

in informal learning environments: a field experiment

in a mathematics exhibition. Computers and Education,

October, Volume 79, pp. 59-68.

Togou, M. A. et al., July 2018. Raising Students' Interest in

STEM Education via Remote Digital Fabrication: An

Irish Primary School Case Study. Palma de Mallorca,

Spain, s.n.

Zou, L., Trestian, R. and Muntean, G.-M., 2018. E3DOAS:

Balancing QoE and Energy-Saving for Multi-Device

Adaptation in Future Mobile Wireless Video Delivery.

IEEE Transactions on Broadcasting, 63(1).

Water Cycle in Nature – An Innovative Virtual Reality and Virtual Lab: Improving Learning Experience of Primary School Students

309