Atomic Structure Interactive Personalised Virtual Lab:

Results from an Evaluation Study in Secondary Schools

Ioana Ghergulescu

, Arghir-Nicolae Moldovan

Cristina Hava Muntean

and Gabriel-Miro Muntean

Adaptemy, Dublin, Ireland

School of Computing, National College of Ireland, Dublin, Ireland

School of Electronic Engineering, Dublin City University, Dublin, Ireland

Keywords: Virtual Labs, Personalisation, STEM Education, Inquiry-based Learning, Evaluation.

Abstract: Virtual labs are increasingly used both as an alternative to physical labs or as a complementary technology

enhanced (TEL) solution for STEM education. Virtual labs enable students to conduct experiments in a

controlled environment at their own pace. However, despite much research on personalisation and adaptation

in the TEL area, most virtual labs that have been developed lack personalisation features. This paper presents

results from a study with 78 secondary school students, aimed at evaluating an interactive personalised virtual

lab called Atomic Structure. The virtual lab integrates personalisation, interactive experimentation, videos, e-

assessment and gamification, to provide an engaging environment for learning chemistry concepts related to

atoms, isotopes and molecules. The evaluation study followed a multi-dimensional methodology to assess the

effectiveness of the virtual lab in terms of knowledge achievement, learner motivation and usability. The

results show that the experimental group that learned with the virtual lab achieved statistically significant

higher knowledge than the control group that attended a traditional teacher led session. The experimental

group also had higher increase than the control group for different motivation dimensions between the pre

and post questionnaires. The usability results showed that most students found the virtual lab useful, easy to

use and liked/loved its features such as videos, quizzes and interactive atom builder.

1 INTRODUCTION

According to many research and government studies,

there is an ongoing concern related to the low and

decreasing engagement with STEM (Science,

Technology, Engineering and Maths) education as

students are progressing from primary to secondary to

tertiary level (Howard, 2017; Milner-Bolotin, 2018;

Patall et al., 2018). Addressing this issue is of major

interest given the growing need for STEM employees

to support technological innovation and economic

growth (European Comission, 2016; OECD, 2015).

The lack of interest in STEM subjects is very

complex and often students lose interest at a too early

stage due to various contributing factors including

perceived difficulty of STEM subjects (Patall et al.,

https://orcid.org/0000-0003-3099-4221

https://orcid.org/0000-0003-4151-1432

https://orcid.org/0000-0001-5082-9253

https://orcid.org/0000-0002-9332-4770

2018; Shirazi, 2017), negative images of the field and

negative ability and self-efficacy beliefs (van

Aalderen-Smeets and van der Molen, 2018; van Tuijl

and van der Molen, 2016). Among the factors that

were identified to address the issue include adaptive

and personalised learning which was shown to

positively corelate with science performance even on

country level data (Mostafa et al., 2018), inquire-

based learning (Howard, 2017), and remote fab labs

and virtual labs (Potkonjak et al., 2016).

The NEWTON Project (http://newtonproject.eu)

is a large scale EU H2020 innovation action project

that focuses on employing novel technologies in

STEM education in order to increase learner quality

of experience, improve learning process and increase

learning outcomes. Innovative technologies include

Ghergulescu, I., Moldovan, A., Muntean, C. and Muntean, G.

Atomic Structure Interactive Personalised Virtual Lab: Results from an Evaluation Study in Secondary Schools.

DOI: 10.5220/0007767806050615

In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 605-615

ISBN: 978-989-758-367-4

605

Augmented Reality and Virtual Reality (D.

Bogusevschi et al., 2018), virtual teaching and

learning laboratory (Bogusevschi et al., 2019), remote

fabrication labs (Togou et al., 2018), adaptive and

personalised multimedia and multiple sensorial

media (Bi et al., 2018; Moldovan et al., 2016), user

modelling and personalisation (Mawas et al., 2018)

and interactive educational computer-based video

games (Mawas et al., 2018). Different innovative

pedagogical approaches are also deployed as part of

the STEM teaching and learning process such as

flipped classroom, game-based and problem-based

learning (Chis et al., 2018; Muntean et al., 2018; Zhao

et al., 2018).

Virtual labs in particular, have been proposed as

one solution to overcome the costs associated with

traditional labs that are resource intensive and costly

to maintain for schools, as well as a solution to make

practical science education available to online

learners (Lynch and Ghergulescu, 2017a; 2017b).

The goal of a virtual lab is to enable students to create

and analyse their own experiments as well as to repeat

them multiple times at their own pace. However,

despite many studies showing the benefits of adaptive

and personalised learning in both classroom and

online settings, most virtual labs for STEM education

lack personalisation features. Furthermore, there is a

limited number of comprehensive case studies and

experiments that evaluated the virtual labs in terms of

their impact on learner motivation aspects such as

engagement, interest and self-efficacy.

In this context, this paper presents the results of a

study performed in an Irish school involving

secondary school students. The study’s goal was to

evaluate an interactive personalised virtual lab called

Atomic Structure. The 78 students that participated in

the study were divided in two groups: an

experimental group that learned following interaction

with the Atomic Structure virtual lab and a control

group that attended a traditional teacher led session.

The research study followed a multidimensional

methodology that applied knowledge tests and

surveys before and after the learning session in order

to comprehensively assess the impact of the Atomic

Structure virtual lab on learners’ knowledge,

motivation and usability.

The rest of the paper is organized as follows.

Section 2 discusses recent related works on virtual

labs. Section 3 overviews the Atomic Structure

virtual lab. Section 4 presents the research

methodology for the evaluation study. Section 5

presents the results analysis. Section 6 discusses the

main findings and limitations of the study and

concludes the paper.

2 RELATED WORK

While many virtual labs were developed over the

years, most of them targeted third-level education

rather than secondary school education, although

universities typically have more resources and better

physical laboratories and equipment. Moreover, this

is despite the fact that learners’ disengagement from

the STEM area starts during secondary level

education in many countries when students start

choosing which subjects they wish to pursue (Bøe and

Henriksen, 2015; van Aalderen-Smeets and van der

Molen, 2018).

Table 1 presents a summary of some existing

virtual labs and platforms. Several European projects

have focused on virtual labs. The Go-Lab project (de

Jong et al., 2014) has created a platform that enables

educators to host and share with other users virtual

labs, apps and inquiry learning spaces. The VccSSe

project (Gorghiu, 2009) created a virtual community

collaborating space for science education that

provided virtual labs and training materials in

physical laws including simulation-based exercises.

The GridLabUPM (Fernández-Avilés et al., 2016)

platform hosts a number of virtual laboratories that

offers students practical experiences in the fields of

electronics, chemistry, physics and topography. The

BioInteractive (HHMI, n.d.) platform provides

science education resources including activities,

videos and interactive media (i.e., virtual labs, click

& learn, interactive videos, 3D models, short

courses). Other virtual labs / platforms include the

Gizmos mathematics and science simulations

(ExploreLearning, n.d.), Chemistry Lab and Wind

Energy Lab (Migkotzidis et al., 2018),

ChemCollective (Yaron et al., 2010), Open Source

Physics (Christian et al., 2011), and Labster (Stauffer

et al., 2018).

Most of these virtual labs offer simulation-based

exercises, interactive activities and online tutorials to

assist the student in their learning journey. The online

tutorials and the multimedia educational resources are

suitable to present the theoretical aspects, while the

interactive activities and simulation-based exercises

are important in achieving the practical skills and in

understanding the phenomena / concepts. While

virtual labs offer students a chance to practice their

all-important practical skills in a safe environment,

most virtual labs lack personalization and adaptation

features, and neglect inclusive education. Many

virtual labs have also been criticised for over

simplification of experiments, with the result that

students do not learn all the necessary skills

associated with specific exercises.

CSEDU 2019 - 11th International Conference on Computer Supported Education

606

Table 1: Summary of existing virtual labs and platforms.

Virtual Lab / Platform Name Activities and Learning Materials Adaptation and Personalisation

The Go-Lab Project

(de Jong et al., 2014)

Multimedia material, Interactive learning

activities

Gamification, Internationalisation, Inquiry

Learning Spaces

Open Source Physics

(Christian et al., 2011)

Chat, email, virtual reality N/A

VccSSe (Gorghiu, 2009) Interactive learning activities N/A

Bio Interactive (HHMI, n.d.) Activities, videos, interactive media N/A

Gizmos (ExploreLearning, n.d.) Interactive simulations N/A

Chemistry Lab, Wind Energy Lab (Migkotzidis

et al., 2018)

Mini-games Difficulty adjustment

ChemCollective (Yaron et al., 2010) Interactive learning activities N/A

Labster (Stauffer et al., 2018) Simulations-based exercises N/A

A number of research studies have conducted

evaluation studies of virtual labs. Aljuhani et al.,

(2018) evaluated a chemistry virtual lab in terms of

usability and knowledge improvement. The virtual

lab was found to be an exciting, useful, and enjoyable

learning environment during user trials. The main

drawbacks of their study were the low number of

participants and the lack of control and experimental

group.

Migkotzidis et al., (2018) evaluated the

Chemistry and the Wind Energy Lab in terms of

usability, adoption, and engagement with the virtual

labs. The participants expressed a positive opinion

regarding the virtual lab interface and high

engagement rates.

Bogusevschi et al., (2018) evaluated a virtual lab

with 52 secondary school students in terms of

learning effectiveness. The results had shown a

statistically significant improvement in the

experimental group using the virtual lab as compared

to the control group learning using classic teacher-

based approach.

Bellou et al., (2018) did a systematic review of

empirical research on digital learning technologies

and secondary Chemistry education. The results of

the review of 43 studies had shown that the

researchers were mainly interested in the chemistry

topics and to use digital learning technologies for

visualisation and simulations but not in personalising

the learning journey.

Despite much research and development in the

area, there still is a lack of personalised virtual labs

and a need for more comprehensive evaluation

studies that look at the impact of virtual labs from

multiple dimensions such as learner knowledge,

motivation and usability. This study contributes to the

area of research through a comprehensive

multidimensional evaluation study of the Atomic

Structure interactive personalised virtual lab with

secondary school students.

3 ATOMIC STRUCTURE

Atomic Structure is an interactive personalised virtual

lab for secondary levels students, that teaches abstract

scientific concepts such as the structure of atoms,

bonding of molecules, gaining and losing electrons,

that can be hard for students to grasp, and difficult for

teachers to present with traditional teaching materials

(Ghergulescu et al., 2018; Lynch and Ghergulescu,

2018). The Atomic Structure virtual lab places the

student in the centre of the learning experience by

implementing personalisation at various layers.

The pedagogical foundations of this virtual lab are

self-directed learning, learning in flow, and inquiry-

based learning. These innovative pedagogies are

beneficial for enabling learners to carry out their own

experiments, analyse and question, and take

responsibility for their own learning (Wang et al.,

2015), while personalisation makes the learning

experience an individual one and keeps the learner

engaged.

Figure 1 shows the models built into the Atomic

Structure virtual lab to enable personalisation and

adaptation. The virtual lab covers concepts such as:

atoms, isotopes and molecules. The learning path is

guided by the Curriculum Model structure and

organisation. For example, a student can only start the

isotopes part of the virtual lab when they meet the

prerequisite of completing the atoms.

Atomic Structure Interactive Personalised Virtual Lab: Results from an Evaluation Study in Secondary Schools

607

Figure 2: Instructional video of atom with embedded sign language translation to support hearing impaired students.

Figure 3: Building an atom of Beryllium.

Figure 1: Adaptation and Personalisation input models:

Pedagogical Model, Curriculum Model, Content Model and

Learner Model.

The Content Model contains various learning

materials and contents available in the virtual labs:

instructional content with videos, e-assessment,

interactivity where students can create and perform

their own experiments through inquiry-based

learning. The Learner Model is updated during the

entire learner journey and includes information about

the learner knowledge, level of self-directness,

motivation (confidence), and special education needs.

Personalisation in the Atomic Structure virtual lab

is implemented at different levels throughout the

entire learning journey. The levels of personalisation

include:

 learning loop-based personalisation;

Adaptation

and

Personalisation

Pedagogical

Model

Curriculum

Model

Content

Model

Learner Model

CSEDU 2019 - 11th International Conference on Computer Supported Education

608

Figure 4: Gamification badge awarded for completing the Atom stage.

 feedback-based personalisation;

 innovative pedagogies-based personalisation

(inquiry-based learning, learning in flow, and self

- directed learning);

 gamification-based personalisation;

 special education needs-based personalisation

(e.g., sign language translation for hearing

impaired students as shown in Figure 2).

Student’s levels of motivation and self-directness are

determined at the beginning of the lab by asking them

to answer few questions displayed on the screen.

These are used to personalise the difficulty level of

questions they receive in the quizzes, what types of

atoms, isotopes and molecules they are given to build,

as well as what type of feedback they will receive. For

example, low and medium motivated students are

restricted to atoms, isotopes and molecules which

have been deemed suitable to each of those levels,

and highly motivated students have access to more

complex atoms, isotopes and molecules.

Figure 3 illustrates the process of building an atom

of boron with the Atomic Structure virtual lab. The

inquiry-based learning phase is offered at the end of

each of the three stages in the form of interactive

atom, isotope and molecule builders.

Once the students master building the suggested

objects, they can freely choose their own objects, and

experiment further within the atom, isotope and

molecule builders. The Atomic Structure virtual lab

also includes gamification elements such as award

badges for completing different stages (see Figure 4).

4 RESEARCH METHODOLOGY

This section details the research methodology for the

case study conducted with the aim to evaluate the

Atomic Structure virtual lab in secondary schools.

4.1 Participants

A total of 78 secondary level students from two

schools in Ireland have participated into the study.

The students were divided in a control group and an

experimental group. The wide majority of students

(i.e., 69 students) were in the 13-15 age group, 6

students were in the 16-18 age group, and 3

participants did not indicate their age group. The

control group had 36 students (23 boys, 11 girls, 2 did

not respond) and the experimental group had 42

students (26 boys, 15 girls, 1 did not respond).

Students from the control group attended a traditional

teacher-led classroom while the students from the

experimental group studied by using the Atomic

Structure virtual lab on computers in the classroom.

The control group was also exposed to the Atomic

Structure virtual lab after the evaluation study.

4.2 Evaluation Process

The evaluation of the Atomic Structure virtual lab

was done following the multi-dimensional

methodology for pedagogical assessment in STEM

technology enhanced learning (Montandon et al.,

2018). The dimensions assessed were: learning

Atomic Structure Interactive Personalised Virtual Lab: Results from an Evaluation Study in Secondary Schools

609

outcome, motivation and learner satisfaction

(usability-based). The flow of the evaluation is

illustrated in Figure 5, while the assessment

procedure is illustrated in Table 2.

Figure 5: Research study workflow.

A description of the research study was given to

participants, and consent and assent forms were

collected before the actual study. Pre-learning

experience surveys were given before and after the

learning experience. The pre-surveys included:

demographics questionnaire, knowledge pre-test and

learner motivation pre-survey for both the control and

experimental group. The learning experience of the

experimental group was a personalised learning

journey through Atomic Structure virtual lab, while

the learning experience of the experimental group

was traditional teacher led-class session. Knowledge

post-tests and Learner motivation post-survey were

given to students from both experimental and control

group. Furthermore, the experimental group filled in

a usability survey.

The knowledge tests contain both multiple choice

and input answer questions. Learner motivation was

assessed through dimensions such as interest, self-

efficacy, engagement, positive attitude and

enjoyment. Interest was assessed through Linkert

scale interest question (Moldovan et al., 2017; Ryan

and Deci, 2000), self-efficacy (confidence) was

assessed following Bandura’s guidelines (Bandura,

2006), while engagement, positive attitude and

enjoyment was assessed using a 5 point Likert scale

(Harmon-Jones et al., 2016). The usability survey

contained questions related to four dimensions

(usefulness, ease of use, ease of learning and

satisfaction), as well as questions where students

were asked to rate tow much they liked different

features on the Atomic Structure virtual lab on a 5-

point Likert scale, as well as open answer questions

to indicate the top three things they liked, top 3 things

they didn’t like, and if they have any comments or

suggestions.

Table 2: Assessment procedure.

Activity Type

Control

Group

Experimental

Group

Demographics Survey Pre-Learning

✓ ✓

Knowledge Pre-test Pre-learning

✓ ✓

Learner Motivation

Pre-survey

Pre-learning

✓ ✓

Atomic Structure

Virtual Lab Session

Learning

✓

Traditional Teacher

Led Session

Learning -

✓

Learner

Motivation Post

survey

Post-learning

✓ ✓

Learner Usability

Survey

Post-learning

✓

Knowledge post-test Post learning

✓ ✓

Interviews Post learning

✓ ✓

5 RESULTS ANALYSIS

5.1 Learning Results

An analysis of the pre-test and post-tests knowledge

was conducted to investigate the impact of the

Atomic Structure virtual lab on students’ learning

outcome. This analysis excluded the participants that

did not answer any question of the pre-test and/or

post-test. This approach was treating the participants

as absent rather than awarding them a score of 0,

which would not be a correct representation of their

knowledge level. Participants with true 0 for pre

and/or post-test (i.e., answered all questions wrong),

were not excluded from the analysis. 11 participants

were excluded from the control group and 2

participants were excluded from the experimental

group. As such, the pre and post-test scores of 25

participants from the control group and 40

participants from the experimental group were

considered for the learning outcomes analysis.

Description of the research study

Collection of consent and assent

forms

Pre-learning experience surveys

Learning experience

Post-learning experience surveys

Interviews

CSEDU 2019 - 11th International Conference on Computer Supported Education

610

Figure 6 presents the average correct response

rates for the control and experimental groups on the

pre and post knowledge tests.

Figure 6: Learning results in terms of mean correct response

rates for the two groups.

The experimental group had a mean correct

response rate of 53.5% (SD = 22.8%) for pre-test and

75% (SD = 22.1%) for post-test, which results in a

21.5% increase. The results of a paired t-test for

dependant groups showed that the post-test results

were statistically significant higher than the pre-test

results for the experimental group at α = 0.05

significance level (t(39) = 5.845, p < 0.001).

The control group had a mean correct response

rate of 48% (SD = 23.1%) for pre-test and 60% (SD =

32.1%) for post-test, which results in a 12% increase.

The results of a paired t-test showed that the post-test

results were statistically significant higher than the

pre-test results for the control group at α = 0.05 (t(24)

= 2.268, p = 0.033).

The experimental group had 5.5% higher correct

response than the control group for pre-test, and 15%

higher for post-test. The results of a t-test for

independent groups showed that the experimental and

control groups had statistically equivalent pre-test

score at α = 0.05 (t(51) = 0.938, p = 0.353). However,

the post-test results for the experimental group were

statistically significant higher than for the control

group at α = 0.05 (t(38) = 2.051, p = 0.047).

5.2 Motivation Results

An analysis of the learner motivation and affective

state questionnaires filled by the students before and

after the session was conducted to investigate the

impact of the Atomic Structure virtual lab on

students’ motivation. This analysis excluded the

participants that did not answer all the questions (i.e.,

4 participants from the control group and 3

participants from the experimental group). The data

from 31 participants from the control group and 39

participants from the experimental group were

considered for the learner motivation analysis.

Figure 7 presents the motivation analysis results.

The percentage of students answering that they are

very or extremely interested in science classes has

increased between the pre and post-session

questionnaires with 18% for the experimental group

and with 9.6% for the control group.

The percentage of students answering that they

are very or extremely confident in being able to solve

science problems and challenges has increased with

28.2% for the experimental group and with 6.5% for

the control group.

The percentage of students answering that they

are very or extremely engaged in science lessons has

increased with 30.8% for the experimental group and

with 9.7% for the control group.

Figure 7: Increase in percentage of learners with high

ratings for different motivation dimensions between the

post and pre-session questionnaires.

The percentage of students that agreed or strongly

agreed that they felt positive during science classes

has increased with 20.5% for the experimental group

and with 3.3% for the control group.

The percentage of students that agreed or strongly

agreed that science classes are interesting has

increased with 20.5% for the experimental group and

with 6.5% for the control group.

53.5%

48%

75%

60%

Pre-t

t-tes

Knowledge Test

Mean Correct Response Rate [%]

Group

Control

Experimental

Learning Results

18%18%

9.6%9.6%

28.2%28.2%

6.5%6.5%

30.8%30.8%

9.7%9.7%

20.5%20.5%

3.3%3.3%

20.5%20.5%

6.5%6.5%

10.3%10.3%

0%0%

Inter

Confide

Eng

Positive

sti

Dimension

Increase [%]

Group

Control

Experimental

Learner Motivation Results

Atomic Structure Interactive Personalised Virtual Lab: Results from an Evaluation Study in Secondary Schools

611

The percentage of students that agreed or strongly

agreed that they enjoy science classes has increased

with 10.3% for the experimental group but did not

change for the control group.

5.2 Usability Results

An analysis of the learner usability questionnaire

completed by the experimental group after interacting

with the Atomic Structure virtual lab was also

conducted. 5 participants were excluded from this

analysis as they did not answer all the questions, thus

the data from 37 participants from the experimental

group were used.

The results analysis showed the following main

findings:

 68.5% of students provided agree or strongly

agree ratings and 11.7% of students provided

disagree or strongly disagree ratings on usefulness

dimension;

 71.2% of students provided agree or strongly

agree ratings and 18% of students provided

disagree or strongly disagree ratings on ease of

use dimension;

 81.1% of students provided agree or strongly

agree ratings and 6.8% of students provided

disagree or strongly disagree ratings on ease of

learning dimension;

 60.4% of students provided agree or strongly

agree ratings and 13.5% of students provided

disagree or strongly disagree ratings on

satisfaction dimension.

Figure 8 also show the percentage of users that

indicated that they liked or loved the different

features / technology of the virtual lab as follows:

86.5% for videos, 83.8% for quiz and reminder of

correct answer after the quiz, 73% for feedback after

the quiz, 64.9% for atom builder, isotope builder and

receiving badges, and 75.7% for reading facts about

atoms and isotopes.

Students also provided subjective feedback. As

part of the negative aspects, they mentioned the fact

that the atom and isotope builders “took a while” to

load and were “sometimes slow”, or “it was slow

loading the build atom game”. One student reported

that had to “load the page as it didn’t work”. Students

were using school’s computers and internet

connection. Another area for improvement suggested

by students was to add more “examples or

instructions to do the exercises”.

As part of the positive aspects, they mentioned “it

is easy to use”, “it is fun”, “it was interesting”, “gets

to the point”, “you can do it yourself”. They reported

on their perceived learning as well: “I have a better

understanding of it now”, “my knowledge of the topic

has improved”, “the videos helped me to learn by

hearing”, “I liked the quiz as I could see for myself

what I had learned”, “it helps you understand easier”,

“I liked how easy it was to understand.”

Figure 8: Percentage of learners that liked/loved the

different features of the Atomic Structure virtual lab.

6 CONCLUSIONS

Virtual labs have been identified as an effective

solution to addressing issues such as the learner’s

disengagement with STEM subjects, expensive

maintenance of physical labs, and availability of

experiential learning to online students. Despite the

research and development effort, most virtual labs

lack personalisation and adaptation, while the

evaluation studies often consider only a small number

of metrics or questions. This paper has presented

results from a comprehensive evaluation of the

Atomic Structure interactive personalised virtual lab,

with secondary school students. The evaluation

applied a multidimensional approach assessing the

virtual lab’s impact on knowledge achievement,

learner motivation, and usability dimensions.

The main conclusion that can be drawn from the

learning analysis is that the students using the Atomic

Structure virtual had a statistically significant higher

knowledge increase than the control group that

86.5%

83.8%

73%

83.8%

64.9% 64.9% 64.9%

75.7%

Builder

Rece

ng Badges

cts

Feature / Technology

Percentage Users [%]

Percentage of users that liked or loved the feature

CSEDU 2019 - 11th International Conference on Computer Supported Education

612

attended the traditional teacher-led session. The main

limitation was the fact that many participants from the

control group had to be excluded from the analysis

(10 participants did not complete the post-test and 1

participant did not complete both pre-test and post-

test). The main observation was that some students

ran out of time at the end and did not manage to

complete the questionnaire before they left for the

next class. Therefore, it is important to better engage

with teachers to ensure they give enough time to

students to complete the forms within the allocated

session timeframe.

The main conclusion that can be drawn from the

motivation analysis is that the Atomic Structure

virtual lab had a higher impact on increasing learner’s

motivation as compared to traditional learning. The

main conclusion that can be drawn from the usability

analysis is that the wide majority of students have

provided agree/strongly agree ratings for the different

usability dimensions (usefulness, ease of use, ease of

learning and satisfaction), and liked/ loved the

features/ technology for the virtual lab.

ACKNOWLEDGEMENTS

This research is supported by the NEWTON project

(www.newtonproject.eu), funded by the European

Union's Horizon 2020 Research and Innovation

programme, Grant Agreement no. 688503.

REFERENCES

Aljuhani, K., Sonbul, M., Althabiti, M., & Meccawy, M.

(2018). Creating a Virtual Science Lab (VSL): the

adoption of virtual labs in Saudi schools. Smart

Learning Environments, 5(1), 16. https://doi.org/

10.1186/s40561-018-0067-9

Bandura, A. (2006). Guide for constructing self-efficacy

scales. Self-Efficacy Beliefs of Adolescents, 307–337.

Bellou, I., Papachristos, N. M., & Mikropoulos, T. A.

(2018). Digital Learning Technologies in Chemistry

Education: A Review. In D. Sampson, D. Ifenthaler, J.

M. Spector, & P. Isaías (Eds.), Digital Technologies:

Sustainable Innovations for Improving Teaching and

Learning (pp. 57–80). Cham: Springer International

Publishing. https://doi.org/10.1007/978-3-319-73417-

0_4

Bi, T., Pichon, A., Zou, L., Chen, S., Ghinea, G., &

Muntean, G.-M. (2018). A DASH-based Mulsemedia

Adaptive Delivery Solution. In Proceedings of the 10th

International Workshop on Immersive Mixed and

Virtual Environment Systems (pp. 1–6). New York, NY,

USA: ACM. https://doi.org/10.1145/3210438.3210443

Bøe, M. V., & Henriksen, E. K. (2015). Expectancy-Value

Perspectives on Choice of Science and Technology

Education in Late-Modern Societies. In E. K.

Henriksen, J. Dillon, & J. Ryder (Eds.), Understanding

Student Participation and Choice in Science and

Technology Education (pp. 17–29). Dordrecht:

Springer Netherlands. https://doi.org/10.1007/978-94-

007-7793-4_2

Bogusevschi, D., Tal, I., Bratu, M., Gornea, B., Caraman,

D., Ghergulescu, I., … Muntean, G.-M. (2018). Water

Cycle in Nature: Small-Scale STEM Education Pilot.

Presented at the EdMedia World Conference on

Educational Media and Technology, Amsterdam, The

Netherlands.

Bogusevschi, Diana, Muntean, C. H., & Muntean, G.-M.

(2019). Teaching and Learning Physics using 3D

Virtual Learning Environment: A Case Study of

Combined Virtual Reality and Virtual Laboratory in

Secondary School. In 30th Annual Conference of the

Society for Information Technology and Teacher

Education (SITE). Las Vegas, NV, USA: AACE.

Chis, A. E., Moldovan, A.-N., Murphy, L., Pathak, P., &

Muntean, C. H. (2018). Investigating Flipped

Classroom and Problem-based Learning in a

Programming Module for Computing Conversion

Course. Educational Technology & Society, 21(4),

232–247.

Christian, W., Esquembre, F., & Barbato, L. (2011). Open

Source Physics. Science, 334(6059), 1077–1078.

https://doi.org/10.1126/science.1196984

de Jong, T., Sotiriou, S., & Gillet, D. (2014). Innovations in

STEM education: the Go-Lab federation of online labs.

Smart Learning Environments, 1(1), 3.

https://doi.org/10.1186/s40561-014-0003-6

European Comission. (2016). Does the EU need more

STEM graduates? (Website) (p. 81). Luxembourg:

Publications Office of the European Union. Retrieved

from https://publications.europa.eu/s/kjn0

ExploreLearning. (n.d.). Gizmos: Math and Science

Simulations That Power Inquiry and Understanding.

Retrieved from https://www.explorelearning.com

Fernández-Avilés, D., Dotor, D., Contreras, D., & Salazar,

J. C. (2016). Virtual labs: A new tool in the education:

Experience of Technical University of Madrid. In 2016

13th International Conference on Remote Engineering

and Virtual Instrumentation (REV) (pp. 271–272).

https://doi.org/10.1109/REV.2016.7444480

Ghergulescu, I., Lynch, T., Bratu, M., Moldovan, A.-N.,

Muntean, C. H., & Muntean, G. M. (2018). STEM

Education with Atomic Structure Virtual Lab for

Learners with Special Education Needs. In

EDULEARN18 Proceedings (pp. 8747–8752). Palma,

Mallorca, Spain: IATED. https://doi.org/10.21125/

edulearn.2018.2033

Gorghiu, G. (2009). VccSSe: Virtual Community

Collaborating Space for Science Education (European

Socrates Comenius 2.1) (pp. 7–16). Retrieved from

http://www.vccsse.ssai.valahia.ro/disvol/VccSSe%20

Dissemination%20Volume%20-%20Article%201.pdf

Atomic Structure Interactive Personalised Virtual Lab: Results from an Evaluation Study in Secondary Schools

613

Harmon-Jones, C., Bastian, B., & Harmon-Jones, E. (2016).

The Discrete Emotions Questionnaire: A New Tool for

Measuring State Self-Reported Emotions. PLoS ONE,

11(8). https://doi.org/10.1371/journal.pone.0159915

HHMI. (n.d.). BioInteractive. Retrieved from

https://www.hhmi.org/biointeractive

Howard, S. (2017). Scientific inquiry: considering

continuity, progression and reasons to focus on primary

and secondary transition (in England). Science Teacher

Education, 79, 24–35.

Lynch, T., & Ghergulescu, I. (2017a). NEWTON Virtual

Labs: Introduction and Teacher Perspective. In 2017

IEEE 17th International Conference on Advanced

Learning Technologies (ICALT) (pp. 343–345).

Timisoara, Romania: IEEE. https://doi.org/10.1109/

ICALT.2017.133

Lynch, T., & Ghergulescu, I. (2017b). Review of Virtual

Labs as the Emerging Technologies for Teaching

STEM Subjects. In INTED2017 Proceedings (pp.

6082–6091). Valencia, Spain: IATED. https://doi.org/

10.21125/inted.2017.1422

Lynch, T., & Ghergulescu, I. (2018). Innovative pedagogies

and personalisation in STEM education with

NEWTON Atomic Structure Virtual Lab. In

Proceedings of EdMedia: World Conference on

Educational Media and Technology (pp. 1483–1491).

Amsterdam, Netherlands: AACE. Retrieved from

https://www.learntechlib.org/primary/p/184368/

Mawas, N. E., Ghergulescu, I., Moldovan, A.-N., &

Muntean, C. H. (2018). Pedagogical based Learner

Model Characteristics. In Ireland International

Conference on Education (IICE-2018) (pp. 138–142).

Dublin, Ireland: Infonomics Society. https://doi.org/

10.2053/IICE.2018.0172

Mawas, N. E., Tal, I., Moldovan, A.-N., Bogusevschi, D.,

Andrews, J., Muntean, G.-M., & Muntean, C. H.

(2018). Final Frontier Game: A Case Study on Learner

Experience. In Proceedings of the 10th International

Conference on Computer Supported Education -

Volume 2: CSEDU (pp. 122–129). Funchal, Madeira,

Portugal: SciTePress. Retrieved from http://www.

scitepress.org/PublicationsDetail.aspx?ID=gOJA0j4ej

Ac=&t=1

Migkotzidis, P., Ververidis, D., Anastasovitis, E.,

Nikolopoulos, S., Kompatsiaris, I., Mavromanolakis,

G., … Hadiji, F. (2018). Enhanced virtual learning

spaces using applied gaming. Presented at the 21st

International Conference on Interactive Collaborative

Learning, Kos Island, Greece.

Milner-Bolotin, M. (2018). Evidence-Based Research in

STEM Teacher Education: From Theory to Practice.

Frontiers in Education, 3. https://doi.org/10.3389/

feduc.2018.00092

Moldovan, A.-N., Ghergulescu, I., & Muntean, C. H.

(2016). VQAMap: A Novel Mechanism for Mapping

Objective Video Quality Metrics to Subjective MOS

Scale. IEEE Transactions on Broadcasting, 62(3), 610–

627. https://doi.org/10.1109/TBC.2016.2570002

Moldovan, A.-N., Ghergulescu, I., & Muntean, C. H.

(2017). Analysis of Learner Interest, QoE and EEG-

Based Affective States in Multimedia Mobile Learning.

2017 IEEE 17th International Conference on

Advanced Learning Technologies (ICALT) (pp. 398–

402). Timisoara, Romania: IEEE. https://doi.org/

10.1109/ICALT.2017.93

Montandon, L., Playfoot, J., Ghergulescu, I., Bratu, M.,

Bogusevschi, D., Rybarova, R., & Mawas, N. E.

(2018). Multi-dimensional Approach for the

Pedagogical Assessment in STEM Technology

Enhanced Learning (pp. 378–383). Presented at the

EdMedia + Innovate Learning, Amsterdam,

Netherlands: AACE. Retrieved from https://www.

learntechlib.org/primary/p/184219/

Mostafa, T., Echazarra, A., & Guillou, H. (2018). The

science of teaching science: An exploration of science

teaching practices in PISA 2015. OECD Education

Working Papers, 188. https://doi.org/10.1787/f5bd9

e57-en

Muntean, C. H., El Mawas, N., Bradford, M., & Pathak, P.

(2018). Investigating the impact of a immersive

computer-based math game on the learning process of

undergraduate students. In Proceedings of the 48th

Annual Frontiers in Education Conference (FIE) (pp.

1–7).

OECD. (2015). How is the global talent pool changing

(2013, 2030)? Education Indicators in Focus, 5.

https://doi.org/10.1787/5js33lf9jk41-en

Patall, E. A., Hooper, S., Vasquez, A. C., Pituch, K. A., &

Steingut, R. R. (2018). Science class is too hard:

Perceived difficulty, disengagement, and the role of

teacher autonomy support from a daily diary

perspective. Learning and Instruction, 58, 220–231.

https://doi.org/10.1016/j.learninstruc.2018.07.004

Potkonjak, V., Gardner, M., Callaghan, V., Mattila, P.,

Guetl, C., Petrović, V. M., & Jovanović, K. (2016).

Virtual laboratories for education in science,

technology, and engineering: A review. Computers &

Education, 95, 309–327. https://doi.org/10.1016/j.com

pedu.2016.02.002

Ryan, R. M., & Deci, E. L. (2000). Self-determination

theory and the facilitation of intrinsic motivation, social

development, and well-being. American Psychologist,

55(1), 68–78. https://doi.org/10.1037/0003-066X.55.

1.68

Shirazi, S. (2017). Student experience of school science.

International Journal of Science Education, 39(14),

1891–1912.

https://doi.org/10.1080/09500693.2017.1356943

Stauffer, S., Gardner, A., Ungu, D. A. K., López-Córdoba,

A., & Heim, M. (2018). Labster Virtual Lab

Experiments: Basic Biology. Springer Spektrum.

Retrieved from https://www.springer.com/gp/book/

9783662579954

Togou, M. A., Lorenzo, C., Lorenzo, E., Cornetta, G., &

Muntean, G. M. (2018). Raising Students’ Interest in

STEM Education Via Remote Digital Fabrication: An

Irish Primary School Case Study. In EDULEARN18

Proceedings (pp. 2835–2840). Palma, Mallorca, Spain:

IATED. https://doi.org/10.21125/edulearn.2018.0756

CSEDU 2019 - 11th International Conference on Computer Supported Education

614

van Aalderen-Smeets, S. I., & van der Molen, J. H. W.

(2018). Modeling the relation between students’

implicit beliefs about their abilities and their

educational STEM choices. International Journal of

Technology and Design Education, 28(1), 1–27.

https://doi.org/10.1007/s10798-016-9387-7

van Tuijl, C., & van der Molen, J. H. W. (2016). Study

choice and career development in STEM fields: an

overview and integration of the research. International

Journal of Technology and Design Education, 26(2),

159–183. https://doi.org/10.1007/s10798-015-9308-1

Wang, J., Guo, D., & Jou, M. (2015). A study on the effects

of model-based inquiry pedagogy on students’ inquiry

skills in a virtual physics lab. Computers in Human

Behavior, 49, 658–669. https://doi.org/10.1016/j.chb.

2015.01.043

Yaron, D., Karabinos, M., Lange, D., Greeno, J. G., &

Leinhardt, G. (2010). The ChemCollective—Virtual

Labs for Introductory Chemistry Courses. Science,

328(5978), 584–585. https://doi.org/10.1126/science.

1182435

Zhao, D., Chis, A., Muntean, G. M., & Muntean, C. H.

(2018). A Large-Scale Pilot Study on Game-Based

Learning and Blended Learning Methodologies in

Undergraduate Programming Courses. In

EDULEARN18 Proceedings (pp. 3716–3724). Palma,

Mallorca, Spain: IATED. https://doi.org/10.21125/

edulearn.2018.0948

Atomic Structure Interactive Personalised Virtual Lab: Results from an Evaluation Study in Secondary Schools

615