Using the Robot-Assisted Attention-Engagement-Error-Feedback-

Reflection (AEER) Pedagogical Design to Develop Machine Learning

Concepts and Facilitate Reflection on Learning-to-Learn Skills:

Evaluation of an Empirical Study in Hong Kong Primary Schools

Siu-Cheung Kong

1,2 a

and Yin Yang

Artificial Intelligence and Digital Competency Education Centre, The Education University of Hong Kong, HKSAR, China

Department of Mathematics and Information Technology, The Education University of Hong Kong, HKSAR, China

Keywords: Learning-to-Learn, Machine Learning, Neuroscience-Informed Pedagogical Framework, Primary Schools,

Robots.

Abstract: This paper presents the results of an empirical study that aimed to evaluate the effectiveness of using robots

to teach machine learning concepts to primary school students and consolidate their reflection on learning-to-

learn skills. The pedagogical design of this study was based on the neuroscience-informed Attention-

Engagement-Error-feedback-Reflection (AEER) framework. The study involved 87 Grade 5 students from

Hong Kong. Data collection included pre- and post-tests on machine learning concepts, as well as pre- and

post-questionnaires on learning-to-learn skills based on the AEER framework. The findings suggest that the

use of purposely designed robots for understanding machine learning significantly enhanced primary school

students’ understanding of machine learning concepts. Further, it can facilitate students’ reflection on their

learning-to-learn skills, which have been nurtured over their years of study period, thereby effectively

preparing them for the transition to secondary school education. The paper concludes with a discussion of the

findings and provides potential directions for future research.

1 INTRODUCTION

Artificial intelligence (AI), and in particular machine

learning, is transforming the global world. The

promotion of AI literacy to all young learners in their

schooling can empower them to have an identification

as part of the future AI society. In this context, the

experiences from teaching machines to learn using

supervised learning and/or reinforcement learning

offer young learners’ unique opportunities to reflect

on their own learning skills.

AI education is pivotal in nurturing students into

educated citizens equipped to thrive in an AI-

prevalent future (Kong et al., 2022; Kong et al.,

2023). Learning-to-learn skills, regarded as the

capacity of individuals to self-regulate, monitor, and

control their learning activities (Cornford, 2002), are

of crucial importance in the contemporary AI-driven

society.

https://orcid.org/0000-0002-8691-3016

https://orcid.org/0000-0002-9966-248X

Despite the need for consolidating learning-to-

learn skills in education (e.g. Vainikainen et al.,

2015), very few studies have explored using robots in

machine learning to engage young students and

connecting machine learning to facilitate reflection

on learning-to-learn (Martin et al., 2023; Kong &

Yang, 2023). This study aimed to conduct an

empirical study on the application of robots, which

are purposely designed for uncovering the black box

of machine learning, to develop Hong Kong senior

primary students’ machine learning concepts and to

facilitate the consolidation of learning-to-learn skills,

based on their past years of study period in schools.

The following research questions were addressed in

this study: Research question 1: To what extent do

primary students enhance their understanding of

machine learning concepts through the pedagogical

activities? Research question 2: To what extent do

primary students enhance the consolidation of

learning-to-learn skills?

Kong, S. and Yang, Y.

Using the Robot-Assisted Attention-Engagement-Error-Feedback- Reﬂection (AEER) Pedagogical Design to Develop Machine Learning Concepts and Facilitate Reﬂection on

Learning-to-Learn Skills: Evaluation of an Empirical Study in Hong Kong Primary Schools.

DOI: 10.5220/0012505700003693

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Computer Supported Education (CSEDU 2024) - Volume 2, pages 155-162

ISBN: 978-989-758-697-2; ISSN: 2184-5026

155

2 LITERATURE REVIEW

2.1 Using Robots to Support Primary

Students’ Machine Learning

Machine learning is a subfield of AI, which has

complex and abstract concepts that can be hard for

young students like primary school students to

understand (Zimmerman, 2018). Despite the inherent

difficulty of teaching these concepts to young learners

like primary school students, there is a growing interest

in integrating machine learning education into K-12

educational settings, aiming to prepare young learners

for an AI-infused future (Lin et al., 2020).

One of the frequent challenges for educators

introducing machine learning-related content is the

complexity of technical terms and the need to present

these terms in a manner that engages students and

sustains their interest (Kong & Yang, 2023). In

response, several user-friendly platforms such as

Teachable Machine and Dancing with AI have been

adopted to engage beginners who are interested in

machine learning. However, these platforms often fall

short of explaining the algorithms that drive the

machines (Voulgari et al., 2021).

Given that the integration of machine learning in

K-12 education is a relatively new area of research

focus, there is a need for empirical studies that

employ innovative approaches to enrich this

developing domain (Sanusi et al., 2023). One

promising approach is to help students understand

machine learning concepts using robots. Various

studies have used robots to elucidate AI and machine

learning concepts to students (Al Hakim et al; 2022;

Olari et al., 2021; Williams et al., 2019). For instance,

the Preschool-Oriented Programming (PopBots)

Platform, which includes a social robot toolkit

comprising a smartphone, LEGO blocks, motors, and

sensors, was used to help preschool students know

machine learning (Williams et al., 2019). This

platform allowed the kindergarten students to talk to

the robots and use a block-based interface to code.

Similar studies have also been explored the use of

coding to create applications with simulated robots

(e.g. Olari et al., 2021). Although the block-based

approach is child-friendly, it does not necessarily

facilitate children’s understanding of the underlying

algorithms of machine learning (Jatzlau et al., 2019).

In our present study, we utilised AlphAI robots

(https://learningrobots.ai/?lang=en) to help non-

programming senior primary school students in Hong

Kong to understand machine learning concepts. This

platform is designed purposely to uncover the black

box of machine learning. The system presents

graphical visualisations to assist students in

understanding the underlying algorithms, such as K-

nearest neighbours, supervised learning using deep

learning approach, and reinforcement learning.

2.2 Facilitating Learning-to-Learn

Skills in Machine Learning

Learning-to-learn skills, also known as metacognitive

skills, are crucial for fostering students’ independence

and enhancing problem-solving and critical thinking

abilities (Cornford, 2002). When students are in their

early stages of education, they already possess these

fundamental learning-to-learn abilities (McCombs,

1991). Students are encouraged to maintain

concentration on learning, engage in practical

activities, actively conduct hands-on activities,

engage in peer discussions, correct errors, and reflect

on their learning experiences through K-12 education

settings (Jocz et al., 2014).

Although it is common for primary school

students to participate in activities that promote the

development of these skills, there is a lack of

instructional methods that specifically target active

reflection on the significance of learning-to-learn

skills (Ashford & DeRue, 2012).

Existing research on the use of robots in machine

learning education primarily focused on students’

perception of machine learning, their motivation, and

engagement (Burgsteiner et al., 2016; Kammer et al.,

2011; Olari et al., 2021; Williams et al., 2019).

However, the extent to which the teaching of machine

learning concepts using purposely designed robots can

facilitate the reflection and consolidation on learning-

to-learn skills of young students is still unknown.

Using robots in machine learning education offers

a unique opportunity to senior primary school

students to reflect on learning-to-learn skills.

Moreover, learning machine learning with robots also

involves iterative learning, where students can

progressively improve the robot’s performance, learn

from their mistakes, evaluate their strategies after

receiving or observing feedback, and apply different

approaches (Voulgari et al., 2021). This process can

assist students in understanding the importance of

persistence, repetitive/deliberate practice, and

reflection in learning.

2.3 Neuroscience-Informed

Pedagogical Framework in

Primary AI Education

In the existing literature, most pedagogical

frameworks for AI education in primary schools tend

CSEDU 2024 - 16th International Conference on Computer Supported Education

156

to employ game-based or project-based learning

approaches (Voulgari et al., 2021; Lee et al., 2021).

However, cognitive scientists argue for the creation

of a novel pedagogical framework that fosters deep,

authentic, and collaborative learning, drawing upon

insights from neuroscience (Hardiman, 2012). The

field of neuroscience has provided significant

contributions in understanding the process of human

learning (Immordino ‐ Yang et al., 2007). These

contributions can be effectively utilised to enhance

the educational practises of AI. According to

Jamaludin et al. (2019), neuroscience-informed

pedagogical framework can facilitate the alignment

of teaching tactics and approaches with the inherent

learning processes of the brain. By employing such an

approach, educators can effectively enhance students’

comprehension of AI and machine learning, sustain

their engagement in the learning process, and foster

the acquisition of crucial skills that are essential for

future endeavours. However, neuroscience-informed

pedagogical frameworks can rarely be found in

primary AI education.

Against this backdrop, Kong and Yang (2023)

proposed the Attention-Engagement-Error-

Feedback-Reflection (AEER) pedagogical design,

which is informed by the four fundamental principles

of learning delineated by Dehaene (2020), a

prominent cognitive neuroscientist from France. The

robot-assisted AEER pedagogical design aligns with

findings from the learning sciences and emphasises

the importance of fostering a deeper understanding of

machine learning concepts and seeks to emphasise

and reinforce learning-to-learn skills. While game

and project-based learning can be effective for initial

engagement and exposure to AI concepts, they may

fall short in facilitating the deeper cognitive processes

that AEER targets. For instance, games may capture

attention and engagement, they may not always

provide the specific, targeted feedback necessary for

students to recognize and understand their errors, nor

the scaffolding to guide reflection on why an error

occurred.

3 THE ROBOTS USED IN THIS

STUDY

In this study, the AlphAI robots were used to teach

primary students’ machine learning in a concrete and

precise manner, effectively “opening the black box of

AI” (Learning Robots, n.d.). Each robot is connected

to the AlphAI software (refer to Figure 1). The robot

is packed with sensors (e.g. wide-angle camera,

ultrasound, infrared line tracking sensors), so it can

recognise via an inbuilt camera. The AlphAI software

provides a graphical representation of AI algorithms,

such as K-nearest neighbours (KNN), artificial neural

network (ANN), and reinforcement learning.

Moreover, it allows students to directly control the

robot using arrows on the keyboard or clicking the

arrows on the screen. As the quality of robot training

is also influenced by the robot’s environment, the

playing arena was set to facilitate robot movement.

Figure 2 depicts the specific arena of training robots.

Figure 1: The AlphAI robot and software.

Figure 2: The arena for training AlphAI robots.

4 CURRICULUM DESIGN

4.1 The Attention-Engagement-Error-

Feedback-Reflection (AEER)

Pedagogical Framework

The AEER pedagogical framework (Kong & Yang,

2023) was implemented in this study. To be specific,

in “Attention”, students were directed to identify

important and relevant information, highlight

important and relevant concepts in the worksheets,

and understand the importance of maintaining and

refocusing attention after taking a short break

throughout the learning process.

In “Engagement”, the course stimulated active

engagement by kindling students’ curiosity in

training robots to learn machine learning. The course

promoted peer discussions, provided hands-on

activities, and offered real-time feedback. Students

were invited to observe the robots’ performance and

recorded their observations in the worksheets.

Using the Robot-Assisted Attention-Engagement-Error-Feedback- Reﬂection (AEER) Pedagogical Design to Develop Machine Learning

Concepts and Facilitate Reﬂection on Learning-to-Learn Skills: Evaluation of an Empirical Study in Hong Kong Primary Schools

157

In “Error-feedback”, students were taught to

formulate hypotheses, test them, and make

corrections based on the feedback received. For

instance, they are encouraged to hypothesise whether

larger K values (K can set from 1 to 100) would yield

improved results when using KNN algorithm and test

their hypotheses in training sessions.

In “Reflection”, students were invited to review

their goals or sub-goals, share their reflections with

others, revise plans to improve their training, and

compare machine learning with human learning.

4.2 Machine Learning and

Learning-to- Learn Course Design

The course design covered (1) introduction to AI, (2)

exploration of three concepts of machine learning: K-

nearest neighbours (KNN), artificial neural network

(ANN), and reinforcement learning, and (3)

integration learning-to-learn skills into learning

activities based on the robot-assisted AEER

pedagogical framework.

4.2.1 Introduction to AI

The course provided a foundational understanding of

AI, discussing its definition and exploring different

types of machine learning such as unsupervised

learning, supervised learning, and reinforcement

learning. It also introduced the four main steps

involved in machine learning: defining the problem,

collecting and cleaning data, training the model,

evaluating the model, and making inferential

conclusions.

4.2.2 Exploration of Three Concepts of

Machine Learning

Students learnt three concepts via training AlphAI

robots. (1) K-nearest neighbours (KNN). The AlphAI

software provides an interactive interface that allows

students to delve into the workings of the K-Nearest

Neighbours (KNN) algorithm. As part of the learning

experience, students can observe how robots make

decisions by considering the proximity of their

nearest neighbours. (2) Artificial neural network

(ANN). The structure of an ANN (e.g., input, hidden,

output layers) is visualised in the AlphAI software. In

ANN, nodes are called neurons. The strength of

connections between neurons is represented by lines

in the AlphAI software. (3) Reinforcement learning.

It is a type of machine learning where an agent (in this

case, AlphAI learning robots) learns to make

decisions by performing actions and receiving

feedback from its environment. Every time the robot

moves, it can receive rewards, the value of Level is

determined by the average rewards obtained per

minute. The robot starts with no knowledge of its

environment and begins to take random actions,

learning from the consequences of these actions. The

robot receives a reward when it successfully moves;

whereas it receives a penalty when it gets stuck. The

robot improves its performances via utilising these

rewards and penalties.

4.2.3 Integration Learning-to-Learn Skills

into Learning Activities Based on the

AEER Pedagogical Framework

The following section explains how the AlphAI

robots were incorporated into the learning activities,

with specific focus to each area of the AEER

framework. Regarding “Attention”, the AlphAI

robots are utilized to introduce students to core

principles of supervised learning (KNN, ANN) and

reinforcement learning. These robots use attention

mechanisms, such as image capturing and recognition

to “learn” how to move. To parallel this, learning

activities are crafted to enhance students’ attention.

Worksheets with text hints, images, and illustrations

are utilized to guide students in concentrating on the

task at hand, such as through exercises like naming

the robot or using thought bubbles. In developing

learning-to-learn skills, students are instructed on

setting clear goals and subgoals for their robot

training sessions. They learn to identify vital

information and underscore key machine learning

concepts. In addition, an emphasis is also placed on

the need for relaxation and refocusing.

Regarding “Engagement”, in the concept of

supervised learning, students directly interact with the

learning process through hands-on activities. When

the robots encounter obstacles on the track in KNN

and ANN activities, students are encouraged to

supervise the robots to observe and then moves again.

Students need to explore the effect of adjusting the

value of K in KNN activities. In reinforcement

learning, the robots adapt their actions based on

rewards and penalties. Students need to observe how

the robots learn. In developing learning-to-learn skills,

The teacher(s) enhance students’ engagement by

sparking students’ curiosity to explore new

knowledge, facilitating peer discussions, guiding

hands-on activities, observing, and recording.

Students are encouraged to engage in deliberation

until they understand the reasons of success and

sources of obstacles when they supervise the robots

to learn, and these activities serve as foods for

CSEDU 2024 - 16th International Conference on Computer Supported Education

158

thoughts in reflecting how students learn with open-

minded observation and engagement in their daily

learning.

Regarding “Error-feedback”, robots encounter

obstacles in moving forward such as hitting a wall and

getting stuck in a corner. Students need to supervise

the robots to act appropriately by providing

opportunities to encounter obstacles and find ways to

overcome. Students need to spend quite a substantial

period in supervising the robots to overcome the

obstacles and these provide opportunities for students

to reflect. In parallel, these activities are designed to

help students reflect on their own learning. Students

observe errors made by robots during training, such

as hitting walls and getting stuck, and adjust their

strategies accordingly. Rather than viewing errors as

failures, they are guided to consider these as learning

opportunities. The teacher guides students in

understanding and rectifying errors made by the

robots and helping them understand the importance of

seeking for feedback. Students are encouraged to

engage in repetitive practices until the obstacles are

resolved in supervising the robots to learn and these

activities serve as foods for thoughts in reflecting how

students learn with perseverance to overcome

learning barriers in daily learning.

The “Reflection” stage is where students

synthesize learning activities and evaluate their

cumulative learning experiences. In the case of robot

training, students can apply different algorithms such

as KNN, ANN, and reinforcement learning, each with

its unique advantages and limitations, to train their

robots for a racing challenge. Through this process,

students came to understand that qualitative data

plays a pivotal role in the robot’s performance. When

faced with obstacles, such data can assist the robot in

performing well in “self-driving”. In developing

learning-to-learn skills, students are guided to

consider various strategies to enhance their

understanding and to seek out new methods for

overcoming conceptual challenges. This reflective

practice is vital for developing a growth mindset.

Students are not merely passive recipients of

knowledge but are actively engaged in figuring out

how to learn and apply new concepts.

5 METHODOLOGY

5.1 Participants

The convenience sampling approach was adopted

(Etikan et al., 2016). The teacher from the selected

school had previously worked closely with the

researchers on other research projects.

A number of 87 Grade 5 students (girls = 43.7%,

boys = 56.3%) from five classes aged between 9 and

10 participated in a two-day workshop. In compliance

with ethical considerations involving participant data,

signed consent forms were obtained from the students

and their parents before initiating the study.

During the workshop, the students were divided

into twelve groups. The grouping was based on

voluntary principles, allowing students to select their

own group members (Rienties et al., 2014). Each

group was provided with an AlphAI robot and a

computer. A teaching tutor from the research team

was assigned to assist two groups.

5.2 Research Procedure

The study included two 2-hour workshops. In the first

workshop, students were introduced to basic concepts

of AI and machine learning (e.g. unsupervised

learning, supervised learning, and reinforcement

learning) through hands-on robot training. The

second workshop let students apply and reflect on the

knowledge gained in the first, training robots to

navigate a new environment while avoiding

obstacles.

Before the first workshop, the students completed

a pre-test of machine learning concepts and a pre-

survey on learning-to-learn skills. The AEER

pedagogical design was employed to deepen the

students’ understanding of supervised learning

(KNN, ANN) and reinforcement learning. The task

involved guiding a robot to move in a clockwise

direction along the track after training (refer to Figure

2). After the second workshop, students completed a

post-test on machine learning concepts, along with a

post-survey.

5.3 Data Collection and Analysis

In this study, the data sources included (1) pre- and

post-tests on machine learning concepts, and (2) pre-

and post-surveys on learning-to-learn using a five-

Likert scale (ranging from strongly disagree 1 to

strongly agree 5).

The instruments used in this study were developed

and validated through the collaborative efforts of

three experts in AI education and two researchers

specialising in metacognition.

The machine learning concept test was designed

to assess students’ conceptual understanding in

machine learning and deep learning (refer to

Appendix I). The test was designed based on Bloom’s

Using the Robot-Assisted Attention-Engagement-Error-Feedback- Reﬂection (AEER) Pedagogical Design to Develop Machine Learning

Concepts and Facilitate Reﬂection on Learning-to-Learn Skills: Evaluation of an Empirical Study in Hong Kong Primary Schools

159

taxonomy and comprised eight items, with a

Cronbach’s alpha of 0.60. To be specific, two items

assessed students’ ability to recognise and recall the

facts related to machine learning procedures. One item

tested students’ comprehension of the machine

learning process. One item evaluated the students’

problem-solving abilities using their understanding of

KNN. Two items accessed students’ evaluative skills

by choosing the right statements about AI and

reinforcement learning. One item focused on analysis,

where students were asked to compare the differences

between supervised and unsupervised learning. The

final item required students to design a plan using the

knowledge they acquired during the workshops.

The questionnaire on learning-to-learn skills

included four dimensions of attention, engagement,

error-feedback, and reflections with 15 items (refer to

Appendix II). The Cronbach’ alpha was above 0.80.

For the data analysis, to address RQ1, paired

samples t-tests were used to compare the pre-and

post-tests. To address RQ2, regarding the students’

learning-to-learn skills in the context of AI education,

the paired samples t-test was used.

6 RESULTS

6.1 Understanding Machine Learning

Concepts

A paired samples t-test was conducted to determine

whether there were statistically significant

differences in the scores of conceptual understating of

machine learning in pre- and post-tests. Table 1

shows the descriptive data of eight items of the test.

A significant difference was observed between pre-,

and post-test, M

diff

= 0.98, 95% CI [0.52, 1.44], p <

0.001).

Table 1: Descriptive data and paired samples t-test of

machine learning concepts.

Dimensions Pre Post Paired samples t-test

M SD M SD t

Item1 0.33 0.47 0.60 0.49 4.01***

Item2 0.17 0.38 0.45 0.50 4.40***

Item3 0.67 0.47 0.70 0.46 0.75

Item4 0.51 0.50 0.64 0.48 1.98*

Item5 0.25 0.44 0.39 0.49 2.25*

Item6 0.34 0.48 0.54 0.50 2.56**

Item7 0.31 0.47 0.63 0.49 5.32***

Item8 0.16 0.37 0.44 0.50 4.26***

Total Grade 2.75 1.51 3.72 1.81 4.22***

Note: M = mean, SD = standardised deviation

* p < 0.05, ** p < 0.01, *** p < 0.001

6.2 Facilitating Reflection on

Leaning-to-Learn Skills

Paired samples t-tests were conducted to determine

whether there were statistically significant

differences in the pre-and post-surveys. Table 2

shows the descriptive data that students learning-to-

learn skills improved in terms of four dimensions:

attention (M

diff

= 0.43, 95% CI [0.25, 0.60], p < .001),

engagement (M

diff

= 0.31, 95% CI [0.14, 0.47], p <

.001), error-feedback (M

diff

= 0.23, 95% CI [0.04,

0.42], p < 0.05), and reflection (M

diff

= 0.28, 95% CI

[0.11, 0.44], p < 0.001) (refer to Figure 3). Overall,

there was a statistically significant increase in

students’ learning-to-learn skills from the pre-survey

to the post-survey (M

diff

= 0.11, 95% CI [0.03,0.04],

p < 0.05).

Table 2: Descriptive data and paired samples t-test of

Leaning-to-learn survey.

Dimensions Pre Post Paired samples t-test

M SD M SD t

Attention 3.59 0.78 4.02 0.68 4.86***

Engagement 3.71 0.69 4.01 0.73 3.74***

Error-feedback 3.72 0.74 3.95 0.73 2.45*

Reflection 3.64 0.74 3.92 0.66 3.38***

Average 3.67 0.60 3.97 0.62 4.56***

Note: M = mean, SD = standardised deviation

* p < 0.05, ** p < 0.01, *** p < 0.001

Figure 3: Students’ perceptions of attention, engagement,

error-feedback and reflection between pre- and post-survey.

7 DISCUSSION AND FUTURE

WORK

This study aimed to address the current research gaps

by focusing on adopting the neuroscience-informed

AEER pedagogical design to enhance primary

students’ machine learning and facilitate students to

actively reflect on their learning-to-learn skills. The

results of this study showed the positive effectiveness

of using robots underpinned by the AEER

pedagogical design in enhancing primary students

CSEDU 2024 - 16th International Conference on Computer Supported Education

160

understanding of machine learning concepts and

facilitating learning-to-learn skills gained from

pervious studying periods. The findings are in line

with prior research, where children showed interest in

investigating why the robot failed to learn efficiently

(Lin et al., 2020; Olari et al., 2021). Through

observing and reflecting on the performance of

robots, students can learn machine learning in a fun

way. In addition, training robots to reflect on

learning-to-learn deepen senior primary students’

metacognitive skills. Thereby there is a potential to

effectively preparing primary students for the

transition to secondary school education, which needs

more metacognitive skills. Although this empirical

study is promising in integrating machine learning

and learning-to-learn skills into primary education,

further exploration is needed.

The study has contributed to both teaching and

research. First, this study provides a novel

pedagogical design that can help teachers make their

instruction more effective and engaging in machine

learning using robots. Furthermore, the study

highlights the potential of the AEER as a guide for

teaching strategies, especially in promoting learning-

to-learn skills in machine learning. By integrating this

model into teaching, educators may be able to foster

a deeper understanding of the subject matter and

enhance students’ metacognitive skills. Second, this

study provides a foundation for further research in

developing primary students’ machine learning

concepts and facilitating learning-to-learn skills

through the robot-assisted AEER. This research fills

an existing gap in the literature and enriches the

practices of using robots to teach machine learning at

the primary school level. Furthermore, the positive

results from the AEER model implementation

indicate its potential for refinement across other

subjects.

The study acknowledges three primary

limitations. First, the curriculum design has

incorporated machine learning with learning-to-learn

into primary education via two 2-hour workshops.

While this approach was novel, it may pose

challenges for primary-aged children due to the

length and difficulty. Even though we used comics

and other visual aids to engage students in our

worksheet designs, which were based on the

characteristics of young learners, future curriculum

designers should consider a variety of tactics to

maintain students’ attention and involvement during

each two-hour workshop.

Second, the relatively low Cronbach’s alpha of

0.60 in machine learning concept tests suggests the

need for improving its reliability. Future studies will

consider refining the test items or increase the number

of items to improve its internal consistency.

Third, the AEER pedagogical design still needs

further iterative refinement. By analysing the robots’

performance and training robots, students were able

to identify the differences and similarities between

machine learning and human learning, while how to

further help senior primary students reflect on

learning-to-learn, especially for receiving feedback to

overcome the obstacles in their learning need to be

further discussed. Future research could include

longitudinal research to investigate the long-term

impacts of the pedagogical design on students’

understanding of machine learning concepts and

development of learning-to-learn skills.

ACKNOWLEDGEMENTS

The authors would like to acknowledge this artificial

intelligence literacy project with the funding support

from the Li Ka Shing Foundation and the Research

Grants Council, University Grants Committee of the

Hong Kong Special Administrative Region, China

(Project No. EdUHK CB357).

REFERENCES

Al Hakim, V. G., Yang, S.-H., Liyanawatta, M., Wang, J.-

H., & Chen, G.-D. (2022). Robots in situated learning

classrooms with immediate feedback mechanisms to

improve students’ learning performance. Computers

and Education, 182, 104483–. https://doi.org/10.1016/

j.compedu.2022.104483

Ashford, S. J., & DeRue, D. S. (2012). Developing as a

leader: The power of mindful engagement.

Organizational Dynamics, 41(2), 146–154.

Burgsteiner, H., Kandlhofer, M., & Steinbauer, G. (2016).

IRobot: teaching the basics of Artificial Intelligence in

high schools. In Proceedings of the Thirtieth AAAI

Conference on Artificial Intelligence (pp. 4126–4127).

AAAI Press. USA.

Cornford, I. R. (2002). Learning-to-learn strategies as a

basis for effective lifelong learning. International

Journal of Lifelong Education, 21(4), 357-368.

International Publishing. https://doi.org/10.1007/978-

3-31994640-5_13.

Dehaene, S. (2020). How we learn: The new science of

education and the brain. Penguin UK.

Etikan, I., Musa, S. A., & Alkassim, R. S. (2016).

Comparison of convenience sampling and purposive

sampling. American Journal of Theoretical and Applied

Statistics, 5(1), 1–4.

Hardiman, M. M. (2012). The brain-targeted teaching

model for 21st-century schools. Corwin Press.

Using the Robot-Assisted Attention-Engagement-Error-Feedback- Reﬂection (AEER) Pedagogical Design to Develop Machine Learning

Concepts and Facilitate Reﬂection on Learning-to-Learn Skills: Evaluation of an Empirical Study in Hong Kong Primary Schools

161

Immordino‐Yang, M. H., & Damasio, A. (2007). We feel,

therefore we learn: The relevance of affective and social

neuroscience to education. Mind, Brain, and Education,

1(1), 3–10.

Jamaludin, A., Henik, A., & Hale, J. B. (2019). Educational

neuroscience: Bridging theory and practice. Learning:

Research and Practice, 5(2), 93–98.

Jatzlau, S., Michaeli, T., Seegerer, S., & Romeike, R.

(2019). It’s not magic after all–machine learning in

snap! using reinforcement learning. In 2019 IEEE

Blocks and Beyond Workshop (B&B) (pp. 37–41).

IEEE. https://doi.org/10.1109/BB48857.2019.8941208

Jocz, J. A., Zhai, J., & Tan, A. L. (2014). Inquiry learning

in the Singaporean context: Factors affecting student

interest in school science. International Journal of

Science Education, 36(15), 2596–2618.

Lee, S., Mott, B., Ottenbreit-Leftwich, A., Scribner, A.,

Taylor, S., Park, K., ... & Lester, J. (2021, May). AI-

infused collaborative inquiry in upper elementary

school: A game-based learning approach. In

Proceedings of the AAAI Conference on Artificial

Intelligence, 35(17), 15591–15599.

Lin, P., Van Brummelen, J., Lukin, G., Williams, R., &

Breazeal, C. (2020). Zhorai: Designing a conversational

agent for children to explore machine learning

concepts. In Proceedings of the AAAI Conference on

Artificial Intelligence, 34(09), 13381–13388.

Martin, M., Chevalier, M., Burton, S., Bonvin, G.,

Besançon, M., & Deneux, T. (2023). Effects of

introducing a learning robot on the metacognitive

knowledge of students aged 8-11, Proceedings

Robotics in Education, Cyprus. https://doi.org/10.1007/

978-3-031-38454-7_15

McCombs, B. L. (1991). Motivation and lifelong learning.

Educational Psychologist, 26(2), 117–127.

Kammer, T., Brauner, P., Leonhardt, T., & Schroeder, U.

(2011). Simulating LEGO mindstorms robots to

facilitate teaching computer programming to school

students. In Proceedings of the 6th European

conference on Technology enhanced learning: towards

ubiquitous learning (pp. 196–209). Springer-Verlag.

Kong, S.-C., Cheung, W. M.-Y., & Tsang, O. (2023).

Evaluating an artificial intelligence literacy programme

for empowering and developing concepts, literacy and

ethical awareness in senior secondary students.

Education and Information Technologies, 28(4), 4703-

4724. https://doi.org/10.1007/s10639-022-11408-7

Kong, S.-C., Cheung, W. M.-Y., & Zhang, G. (2022).

Evaluating artificial intelligence literacy courses for

fostering conceptual learning, literacy and

empowerment in university students: Refocusing to

conceptual building. Computers in Human Behavior

Reports, 7, 100223.

Kong, S.-C., & Yang, Y. (2023). Designing and evaluating

an attention-engagement-error-reflection (AEER)

approach to enhance primary school students artificial

intelligence literacy and learning-to-learn skills: A pilot

study. In Proceedings of the 31st International

Conference on Computers in Education Volume II

(pp.394-399), Shimane, Japan.

Olari, V., Cvejoski, K., & Eide, Ø. (2021, May).

Introduction to machine learning with robots and

playful learning. In Proceedings of the AAAI

Conference on Artificial Intelligence, 35(17), 15630–

15639.

Rienties, B., Alcott, P., & Jindal-Snape, D. (2014). To let

students self-select or not: That is the question for

teachers of culturally diverse groups. Journal of Studies

in International Education, 18(1), 64–83.

Sanusi, I. T., Oyelere, S. S., Vartiainen, H., Suhonen, J., &

Tukiainen, M. (2023). A systematic review of teaching

and learning machine learning in K-12 education.

Education and Information Technologies, 28(5), 5967–

5997. https://doi.org/10.1007/s10639-022-11416-7

Learning Robots. (n.d.). The concept AlphAI, the robot

made by Learning Robots. Discover AI. Retrieved

November 23, 2023, from https://learningrobots.ai/the-

concept/?lang=en

Vainikainen, M. P., Wüstenberg, S., Kupiainen, S.,

Hotulainen, R., & Hautamäki, J. (2015). Development

of learning to learn skills in primary school.

International Journal of Lifelong Education, 34(4),

376–392.

Voulgari, I., Zammit, M., Stouraitis, E., Liapis, A., &

Yannakakis, G. (2021). Learn to machine learn:

Designing a game-based approach for teaching

machine learning to primary and secondary education

students. Proceedings of the 20th Annual ACM

Interaction Design and Children Conference, 593–598.

https://doi.org/10.1145/3459990.3465176

Williams, R., Park, H. W., & Breazeal, C. (2019). A is for

artificial intelligence the impact of artificial intelligence

activities on young children’s perceptions of robots.

Conference on Human Factors in Computing Systems -

Proceedings.

https://doi.org/10.1145/3290605.3300677

Zimmerman, M. (2018). Teaching AI: Exploring new

frontiers for learning. International Society for

Technology in Education.

APPENDIX

https://docs.google.com/document/d/1F7mclwYP0s

Yj3T9Fg9IkAmG6SBkN4Bt2uIxISLCLJtc/edit?usp

=sharing

CSEDU 2024 - 16th International Conference on Computer Supported Education

162