Chatbot-mediated Learning: Conceptual Framework for the Design

of Chatbot Use Cases in Education

Stefan Sonderegger and Sabine Seufert

Institute for Educational Management and Technologies, University of St.Gallen,

St. Jakob-Strasse 21, 9000 St.Gallen, Switzerland

Keywords: Chatbots in Education, Chatbot-mediated Learning, Conversational Learning, Pedagogical Conversational

Agents, Tutorial Dialogue.

Abstract: While chatbots or conversational agents are already common in many business areas, e.g. for customer support,

their use in the education sector is still in its infancy. Chatbots might take over the role of a teacher, tutor,

conversational partner, learning analyst, team member, support assistant, or recommender system. Within

these different roles, chatbots can enhance learning and inherently address many requirements and success

factors for learning. The scalability and adaptiveness of conversational AI allow an individualised learning

support for all learners combined with collaboration opportunities and thus more equality in education. In this

context, the paper at hand discusses this pedagogical potential of chatbots in different roles and social settings

resulting in a conceptual framework for the understanding and design of chatbot use cases in education. Based

on success factors for learning derived from established learning theories and reports, core attributes and goals

of chatbot learning are deducted within three pedagogical domains of individual, social and analytic chatbot

learning. By combining this pedagogical dimension with a technological and content dimension, the presented

conceptual framework provides an overview of possibilities of how chatbots in education can be used and

designed.

1 INTRODUCTION

The educational use of chatbots – chatbot-mediated

learning – is an emerging research field in education.

Chatbots are artificial intelligence (AI) based

programs that aim to simulate human conversation

(Garcia Brustenga et al., 2018). It can be assumed that

such conversational agents are also suitable for

certain tasks in the field of education and learning. To

date, however, chatbots are not yet widely used in

education. The aim of this research paper is set in this

context. To advance the pedagogical implementation

of chatbots in education, it is important to find out

what has already been done, to structure this

knowledge and make it understandable for

pedagogical practice. This specific research stream is

interdisciplinary and addressed by researchers from

fields like computer science, education, linguistics,

psychology, and business informatics. This leads to

complementary but different research procedures and

evaluation approaches (Hobert, 2019).

From an educational perspective it seems

essential to further identify what pedagogical uses

and capabilities a chatbot has in an educational

context. In this circumstance, it is also relevant to

discuss the different roles and settings in which a

chatbot can be useful (e.g individual or team learning

situations) in relation to success factors for learning.

In addition, for the design of chatbot use cases, it is

fundamental to consider the technological maturity

and integration into educational systems to enhance

the chatbot's capabilities.

Research in this direction seems sensible for

several reasons. On the one hand, there are promising

pedagogical possibilities and on the other hand, one

can address future skills and competences. Visible

learning and the individual support of learners by

teachers or human tutors are somewhat neglected due

to large course sizes and an emerging number of

online learning scenarios. Both, learning theories and

empirical learning studies suggest the relevance of

learner-centred learning, individual support, a culture

of inquiry, continuous feedback and monitoring,

formative feedback, and so on (Bransford et al.,

2000), (Hattie & Yates, 2013). International

frameworks for 21st-century learning suggest that

critical thinking, making judgments and decisions,

clear communication, collaboration, and

Sonderegger, S. and Seufert, S.

Chatbot-mediated Learning: Conceptual Framework for the Design of Chatbot Use Cases in Education.

DOI: 10.5220/0010999200003182

In Proceedings of the 14th International Conference on Computer Supported Education (CSEDU 2022) - Volume 1, pages 207-215

ISBN: 978-989-758-562-3; ISSN: 2184-5026

207

technological awareness are crucial competencies in

the future (ISTE, 2017). Chatbots might support

learners to develop, improve and reflect these

competencies. Furthermore, students work hand in

hand with digital assistants, which becomes standard

in future work activities. The authors of the book

“Human + Machine. Reimagining Work in the Age of

AI” argue that humans need new skills to work with

smart machines and that we need a deeper

understanding of the complementary human-machine

interaction (Daugherty & Wilson, 2018): “Humans

are needed to develop, train, and manage various AI

applications. In doing so, they are enabling those

systems to function as true collaborative partners. For

their part, machines in the missing middle are helping

people to punch above their weight, providing them

with superhuman capabilities, such as the ability to

process and analyse copious amounts of data from

myriad sources in real-time. Machines are

augmenting human capabilities” (p. 6).

Within this paper, we focus on the pedagogical

foundations of human-machine interaction with

chatbots in education and address the following

research questions: RQ1. What pedagogical benefits

and capabilities do chatbots have in an educational

context? RQ2. How can a framework for the use of

chatbots in education be conceptualized? The goal is

to elaborate and communicate the potential of AI-

based chatbots to function as an individual or

collaborative learning partner and to augment

student’s capabilities. Based on the state of research

(section 2) we present a conceptual framework

aiming at providing a pedagogical basis for the

educational use of chatbots (section 3) before we

conclude with final remarks.

2 RESEARCH ON CHATBOTS IN

EDUCATION

Based on the stated research goal, we identify

research on intelligent chatbots in general and

research on chatbots in education as relevant.

2.1 Research on Intelligent Chatbots

While the first chatbot named Eliza was already

developed over 50 years ago by Weizenbaum (1966),

the major developments have happened in recent

years. Core technologies or components of modern

chatbots like automated speech recognition (ASR),

natural language processing (NLP) and text-to-

speech engines rely on deep learning neural networks

and thus AI technologies. Due to technological

milestones and the increasing attention to AI,

chatbots, also known as conversational agents or

natural dialog systems, are an emerging field of

interest in many areas such as e-commerce, health,

finance, service industries, and education. From a

business perspective chatbots mainly allow to

improve customer service and to reduce service costs,

from a user’s perspective important motivations to

use chatbots are productivity, entertainment, social

factors and novelty interaction (Adamopoulou &

Moussiades, 2020).

Examples of modern and widely known chatbots

are Amazon’s Alexa, Apple’s Siri, Microsoft’s

Cortana or Google Assistant, which can also be

categorized as personal assistants mostly used for

customer service and information acquisition or as a

user interface for mobile devices (Cahn, 2017).

Adamopoulou and Moussiades (2020) offer a detailed

chatbot categorization and differentiate between

informative, conversational or task-based chatbots to

point out the main goal of the chatbot. A further basic

categorization of chatbots can be the knowledge

domain. Some chatbots have one or more specific

knowledge domains, whereas a generic chatbot is

designed to answer any user question. While we refer

to both categories as chatbots, other researchers with

a more technical perspective prefer the umbrella term

‘conversational systems’ and then differentiate

between more domain- or task-specific ‘dialog

systems’ and generic ‘chatbots’ (cf. Chen et al.,

2017). An example for a generic and state-of-the-art

chatbot that has won the Loebner Prize Turing test for

best chatbot several times in recent years is Mitsuku

(https://kuki.ai/). Services like IBM Watson or

Google Dialogflow offer platforms and frameworks

for companies or institutions to build and train

domain-specific chatbots based on transfer learning

techniques. The chatbot already knows how to learn

(pre-trained) and is fed with domain-specific

knowledge and rules for its specific use case. This

allows customization and personalization of the

chatbot based on a given basic structure. Most

chatbots are set on a retrieval-based approach, where

responses are generated based on pre-trained rules

and matched through machine learning classification

tasks. While such retrieval-based models promise

accurate and correct responses in case of a correct

match, they are unable to answer unseen questions or

intents without predefined responses or actions

(Winkler & Söllner, 2018). This problem can be

solved with generative models, the newest generation

chatbots. Generative models do not answer with pre-

defined answers but try to generate their answers

CSEDU 2022 - 14th International Conference on Computer Supported Education

208

based on the context, previous dialogs, and a

pretraining based on real dialogs (Cahn, 2017). The

amount of available dialog data is therefore a key

success factor for underlying deep learning models.

Only intensive training enables a chatbot to recognize

and adapt patterns in human dialogs based on

statistically frequent answers (Spierling &

Luderschmidt, 2018). Retrieval-based models are

considered as more reliable until today due to

simplicity, while generative models are better for text

generation and promise a more real conversation

(Molnar & Szuts, 2018). On the downside, the

generative model can only be as good as its

underlying training data. So, if the data is flawed,

corrupted or biased, so is the chatbot. This may be one

reason why the two approaches are increasingly

combined.

Considering this technical side and development

of chatbots we can draw on recent studies defining

technical dimensions to categorize educational

chatbots (Winkler & Söllner, 2018), (Molnar & Szuts,

2018):

1) Building approaches, where retrieval-based

models are distinguished from generative

models. While the former are based on a set of

predefined responses, using an algorithm to

select the best-matching response, the latter

generates responses based on the input.

2) Input mode of chatbots, in particular the

question of whether speech over text input

might be appropriate for our context and

learning design.

3) Inclusion of contextual information, such as for

example time, location, user information,

learning path data, in order to select the right

responses.

From an ultimate technological perspective, the

goal of a chatbot and consequently the evaluation

focus might be to pass the so-called Turing test (cf.

Turing, 1950), meaning that the optimal chatbot

cannot be distinguished from humans. Cahn (2017)

mentions further perspectives to evaluate

performance: From a user experience perspective

another goal would be user satisfaction, from a

linguistic perspective a goal would be for the chatbot

to speak grammatically correct and meaningful and

from an information retrieval perspective chatbots

should also be evaluated according to the specific

function (Cahn, 2017). In an educational context, this

function and therefore the evaluation perspective

might differ again from case to case but would

additionally include learning outcome, learning

success, and learner motivation.

2.2 Chatbots in Education

In recent years the spread of chatbots and the research

on chatbot development, design, and use has

increased and advanced. Følstad et al. (2020) are

convinced that chatbots are maturing for application

areas including education and may be designed for

individual users or for supporting collaboration.

Previous research on chatbots in education often

focuses on designing messenger-like chatbots but

there might be a lack on generalizable results (Meyer

von Wolff et al., 2020). Winkler and Söllner (2018)

conducted an extensive literature review and

conclude that “the effectiveness of chatbots in

education depends on individual student differences,

the ways of building chatbots, and the chatbot

mediated learning process quality” (p.29). While the

authors consider only few studies that suggest the

potential of chatbots for learning purposes so far, they

also emphasize the great potential of chatbots to

create individual learning experiences for students

and to support teachers. The exploration of this

potential in the field of technology-mediated learning

– chatbot-mediated learning – is a growing and

interdisciplinary research field. It can however, draw

on a rich body of previous research in different

educational research fields around pedagogical

agents and tutorial dialogue systems. Research in this

field suggests that both, support by a tutorial dialogue

agent and collaborative learning support lead to better

learning outcomes than supportless learning (Kumar

et al., 2007).

Compared to traditional intelligent tutoring

systems or pedagogical agents in e-learning

scenarios, chatbots do not only give instructions or

provide feedback, but can also react to individual

intents and create a real personalization and more

importantly, a learner-centred approach (Winkler &

Söllner, 2018). While these technologies can be

integrated and build on each other, chatbots can be

regarded as conversation technologies that have a

more stand-alone character compared to adaptive

learning systems. Chatbots in education can still have

different user interfaces or be embedded in other

systems like a Learning Management System (LMS).

The main difference between chatbots in education

compared to other contexts is probably the integration

or self-storage of learning objects or even learning

paths (Hobert, 2019). Figure 1 illustrates a technical

setup of a chatbot in an educational setting in a high-

level abstraction.

With the ultimate goal to enhance and enable a

learner-centred individual and collaborative learning

setting, chatbots promise to have a positive impact on

Chatbot-mediated Learning: Conceptual Framework for the Design of Chatbot Use Cases in Education

209

Figure 1: Educational Chatbot Model. (cf. Seufert et al., 2021).

student motivation, satisfaction, and learning success

(Winkler & Söllner, 2018). In education practice,

however, the productive use of chatbots is still in its

infancy. Nevertheless, research groups see great

potential of chatbots in education and present

promising use cases for different tasks such as

learning assessments, reflections, language learning,

motivating, mentoring, administration, or

productivity assistant (Garcia Brustenga et al., 2018).

Most studies, that have shown successful

implementations of chatbot learning scenarios (cf.

Dutta, 2017; Goel et al., 2016; Huang et al., 2017),

are based on projects with isolated chatbot tasks, e.g.

to answer frequently asked questions, to handle forum

posts, or to ask questions in language learning

applications. Language learning is a more advanced

application of chatbots in education. Fryer et al.

(2019) summarize that in early research chatbots as

language practice tools were shown to be useful for

advanced and motivated students, but showed

limitations in terms of in- and output quality. More

recent research shows, that the linguistic quality has

improved significantly and that chatbot conversations

are carried on longer but with fewer words and

vocabulary within messages compared to human-

human conversations (Fryer et al., 2019).

Another example of more advanced chatbot use

cases in education is supporting students with course

and administrative information or offering screening

tests via chatbot. This application is already

implemented at universities worldwide but can, at

least until today, be considered more of a customer

service chatbot use case than chatbot-mediated

learning. Within a comprehensive conceptual

framework, such use cases could in a further sense be

assigned to educational recommender systems. These

are seen as electronic systems containing domain

knowledge, learner information, and knowledge of

the teaching strategies which seek to improve

learning (Bodily & Verbert, 2017). A future chatbot

might also combine and integrate the intelligence of

the different student-facing learning analytics

systems distinguished by Bodily and Verbert (2017):

Learning Analytics Dashboards, Educational

Recommender Systems, Educational Data Mining

Systems, Intelligent Tutoring Systems. Another

related approach addressing the integration of AI

technologies in education is cognitive computing in

education, where a cognitive assistant (e.g. a

cognitive bot) would combine different AI services

(Lytras et al., 2019).

As introduced in section 2.1, there is still the

question of how to evaluate educational chatbots in

terms of learning outcomes, learner motivation,

learner satisfaction or other constructs. Winkler and

Söllner (2018) propose a learning taxonomy model

(Anderson, 2001) as a basis for the evaluation of

learning outcomes, further evaluating the influence of

chatbots on self-efficacy and self-regulation skills.

Within 25 studies reviewed in the field of chatbot-

mediated learning, Hobert (2019) identified 7

evaluation objectives (acceptance and adoption,

learning success, motivation, usability, technical

correctness, further psychological factors, and

beneficial effects) and matched the objectives with

the main research procedures identified (Wizard-of-

Oz experiment, technical validation, laboratory

experiment, field experiment). The author emphasizes

that most studies analyse only selected aspects and he

attributes this to the interdisciplinarity of this field

(Hobert, 2019). This fact supports the aim of this paper

to present a comprehensive framework and to provide

a basis for future research projects that evaluate the use

of chatbots beyond single evaluation objectives.

Learner

Input

Speech to text

ASR

Chatbot

Output

Interaction database

User and dialog history, data

and learning analytics

Intent

identification

NLP / NLU

Dialog

management

Sentiment analysis

Response

generation

NLP / NLG

Context database

Domain knowledge, intents

and entities

External data requests

LMS, Web APIs, external databases

Channels: Application, Webservice, Messenger, Social Media, LMS, AR, VR, Social Robot, …

Text to speech

TTS

Sentiment output

Input format:

- Audio

-Text

- Picture / video

- Haptic / various sensors

Output format:

-Audio

-Text

- Picture / video

- Haptic / various actuators

CSEDU 2022 - 14th International Conference on Computer Supported Education

210

3 CONCEPTUAL FRAMEWORK

Within this section, we present a conceptual-level

framework to understand chatbot-mediated learning

and to design pedagogical chatbot use cases. The goal

of the presented framework is to lay out what

pedagogical uses and capabilities a chatbot has in an

educational context. Hence, the presented framework

can offer a foundation for the conceptual and

pedagogical design of chatbot use cases and to

uncover the further potential and limitations of the

underlying technologies within education.

Table 1: Chatbots in Education – Maturity Levels.

TPACK Level 1 Level 2 Level 3

Pedagogy

Individual

chatbot learning

+ Social

chatbot

learning

+ Metacognition

and analytics

Technolog

Simple rule-

based chatbot

+ Supervised

learning AI

+ Sentiment

+ Unsupervised

learning AI /

Generative

Content

Domain

knowledge

Chatbo

Social/context

knowledge

+ Omniscient

Note: Categories based on TPACK model (Koehler & Mishra,

2009)

The structure of the presented framework is based

on the Technological Pedagogical Content

Knowledge (TPACK) model, which is widely used in

educational research. The TPACK model originally

represents a framework for teacher knowledge for

technology integration Koehler and Mishra (2009).

Even though a chatbot should primarily be learner-

centred it also should integrate these same three and

closely connected components of educator

knowledge. Table 1 gives an overview of the three

components by which we study chatbots in education

and in a first step indicate high-level maturity levels.

The theoretical background in section 2 has

shown that present chatbots in education are mainly

designed for personal learning and as conversational

partners or tutors, with an information retrieval

approach based on domain knowledge (maturity level

1-2). Future chatbots might include more

collaborative learning, analytics (cf. Ifenthaler &

Schumacher, 2016) and metacognition functions, use

more advanced AI and generative models, and

additionally retrieve and process social and context

information (maturity level 2-3).

3.1 Pedagogical Perspective and Goals

With a focus on educational maturity, we intend to

substantiate the potential of chatbot learning with

learning theory in the form of connections with

established learning theories and more recent learning

reports and empirical studies (Table 2). A corpus of

learning-related theories (Bandura, 1997; Deci &

Ryan, 2012; Leventhal et al., 1984) and reports

(Bransford et al., 2000; Hattie & Yates, 2013; ISTE,

2017) was chosen based on broad acceptance in

educational research and related based on its key

concepts and constructs for learning success. Table 2

illustrates the pedagogical perspective and theoretical

deduction of core attributes and goals of chatbot

Table 2: Chatbots in Education – Pedagogical Perspective and Goals.

Individual

chatbot learning

Social

chatbot learning

Metacognition &

analytics

Core attributes and goals

of chatbot learning

based on underlying

learnin

theories and re

orts

Personalized and

needs-

ased learnin

Collaboration and

network memor

Learning progress and

formative assessment

Individual learning pace Social embedding Feedback & reflection

Self-determination theory

(Deci & Ryan, 2012)

Autonomy experience

Social relatedness Competence experience

Self-efficacy / social cognitive

theor

(

Bandura, 1997

)

Self-Mastery;

Self-Re

ulation

Role-Modelling;

Verbal

ersuasion

Self-Efficacy;

Feedback

Self-regulation theory

(

Leventhal et al., 1984

)

Self-Reflection Team-Reflection Metacognition;

Monitorin

How to learn

(

Brans

ord et al., 2000

)

Learner centred;

considered learn

ath

Culture of inquiry Assessment centred;

continuous monitorin

Framework 21st century learning

(ISTE, 2017)

Communication skills;

decision making

Collaboration skills;

creativity; empathy

Critical thinking;

Use of technology

Visible learning

(

Hattie & Yates, 2013

)

Feedback;

Self-Verbalisation

Reciprocal teaching

(

dialo

based

)

Formative evaluation;

Meta-co

nitive strate

Chatbot-mediated Learning: Conceptual Framework for the Design of Chatbot Use Cases in Education

211

Table 3: Conceptual Framework for Chatbots in Education.

Dimensions Characteristics (can be met supplementary)

Pedagogical dimensions

Chatbot role in

individual learning setting

Cf. (Garcia Brustenga et al.,

2018)

Support assistant

e.g. research assistant,

FAQ, Nerdybot

Conversational partner

e.g. communication

trainer, tutor

Recommender system

e.g. learning path

recommendations

Learning analyst

e.g. formative

assessment

Chatbot role in

social learning context

Cf. (Garcia Brustenga et al.,

2018)

Teacher

e.g. storytelling, debater,

presenter, teaching

assistant

Team member

e.g. maintain project

documentation, team

support, research

Collaboration enhancer

e.g. connect teams,

structure teamwork

Team analyst

e.g. analyze teamwork

and provide feedback

Learning Analytics

Cf. (Fryer et al., 2019;

Ifenthaler & Schumacher,

2016)

Summative assessment

(ask questions and give

feedback)

Formative assessment

(reflect on learning

progress, continuous

feedback)

Intelligent edu-

recommender system

(continuous learning

process monitoring)

Emotion analytics

(Monitor and analyse

sentiments /emotions

to improve learning)

Technological dimensions

Human-Computer

Interaction Cf. (Spierling &

Luderschmidt, 2018)

Visual based

(e.g. text)

Audio based

(e.g. speech)

Virtual presence

(e.g. virtual agent)

Physical presence

(e.g. robot)

Intelligence

Cf. (Lytras et al., 2019;

Molnar & Szuts, 2018)

Simple rule-based model Retrieval-based model Generative models and

unsupervised learning

models

Social intelligence:

Sentiment analysis,

emotion detection

Embedding / Channel Local application (mobile

/ computer)

Webservice

(on any device)

Social Media

(known channel)

Embedded in LMS

(on any device)

Content dimensions

Contextual data integration

Cf. (Molnar & Szuts, 2018;

Winkler & Söllner, 2018)

Basic domain knowledge

database

Basic context data

(time, location, user)

Personalisation with

conversation history

Personalisation with

learner information

(learner path /grades)

Knowledge base

Cf. (Anderson, 2001)

Factual domain

knowledge

Conceptual and

procedural knowledge

Knowledge

representation

Metacognitive and

social knowledge

learning. The bullet point attributes and goals

included, describe elementary principles and success

factors for learning and create a basis for linking

chatbot learning with concrete didactic goals. They

thus help in the justification of chatbot learning and

as a conceptual basis for concrete goals and their

evaluation.

3.2 Conceptual Design of Chatbot Use

Cases in Education

At the beginning of a project to design and use a

chatbot in education, the following questions are of

central importance in connection with the objective:

• What are the (pedagogical) goals of the chatbot

use? What is the context?

• Which target group does the chatbot address?

• What is the role and what are the tasks of the

chatbot? What is the role of the learner?

• What are the limitations or technological

requirements? What (sensitive) data is used?

• What is the time, personnel and financial

budget?

Since chatbot projects in education, just like

innovations in learning technologies, are often driven

by a technological direction, it is recommended to

combine the pedagogical with the technological

perspective at an early stage to develop a shared new

vision of learning (Dillenbourg, 2016). Based on the

pedagogical perspective from section 3.1 one can

identify the educational setting and define desired

learning conditions and pedagogical goals in order to

answer the questions posed.

Accordingly, in table 3 we lay out the core of the

conceptual framework for the conceptual design of

chatbot use cases in education, based on the

pedagogical perspective from section 3.1 and the

theoretical and technical background from section 2.

The three components of the TPACK model serve as

the main structure. The goal is a collectively

configuration of the pedagogy, technology and

content dimensions. Each dimension has four

CSEDU 2022 - 14th International Conference on Computer Supported Education

212

characteristics that can be met supplementary (e.g. a

chatbot can have the role of a conversational partner

and a learning analyst combined, or be available over

multiple channels).

When planning a chatbot use case in education,

the pedagogical point of view is central together with

the goal of the chatbot use. The role of the chatbot in

an individual or social learning setting already partly

determines the further requirements. At the same

time, the roles or tasks that a chatbot can take on in

practice today are often limited by the technological

possibilities and boundaries. An extensive overview

as well as practical examples of chatbots in education

divided by tasks is provided by Garcia Brustenga et

al. (2018). When it comes to developing a chatbot in

education, Satow (2019) describes the following

development steps:

• Creating the bot concept

• Analysis of real dialogs and questions

• Creation of bot scripts

• Bot training by defining intents

• Bot skills development

• Testing the chatbot

• Optimize in productive use

When designing chatbot dialogs, general and

education specific design principles can be helpful

(cf. Yu et al., 2016). Cahn (2017) describes 'human

imitation strategies' that have proven successful (e.g.

personality development, conversation control,

human errors). And in terms of developing a chatbot

for learning purposes, Smutny and Schreiberova

(2020) offer a list of attributes describing the quality

of an educational chatbot within the categories

teaching (e.g. set goals and monitor learning

progress), humanity (e.g. able to maintain themed

discussion), affect (e.g. entertaining, engaging) and

accessibility (e.g. responds to social cues

appropriately).

From a technological point of view, in addition to

the form of interaction and chatbot intelligence, the

main question is its integration or embedding.

Educational institutions such as schools and

universities as well as educational organisations in

companies often use learning management systems.

Here it is important to clarify whether the chatbot can

be integrated into existing learning platforms or, if

other channels are used, how the learner authenticates

himself to the chatbot, if necessary. This is especially

important if the chatbot is to access not only general

knowledge (factual, conceptual, procedural or social)

but also contextual knowledge about the individual

learner from a content perspective. This can be,

among other things, personal, performance-related or

behavioural data. The data basis and its use result in

limiting factors and requirements, which are

discussed in the following section 3.3.

3.3 Limitations

Depending on the role of the chatbot, data storage or

connection to databases, it is important to clarify the

topics of data protection, data storage, data security,

data integrity and data deletion at an early stage.

Since AI services are often computing power-

intensive and therefore cloud-based, the privacy of

the individual is all the more important (Walsh,

2018). According to (Cahn, 2017), chatbots via

messenger applications or services are problematic

from a privacy perspective, as services such as

Facebook Messenger do not offer end-to-end

encryption by default and cannot guarantee user

identification. At the same time, many chatbots and

services include and process sensitive data (personal

data, images, audio, video). Services such as Amazon

Echo store recordings in the cloud, while many other

services send the data for further processing

unencrypted via APIs. All these factors speak in

favour of in-house development and local data

storage and processing. Nevertheless, researchers

also emphasise advantages of smart assistants or

messaging applications such as a familiar user

interface, no installation, no costs, integration of

games, sharing of media (Smutny & Schreiberova,

2020). Regardless of the technology chosen, there are

ethical issues to discuss. In addition to data

protection, there is an important demand for

explainable AI (XAI), which is particularly important

in the field of education (Gunning, 2017). Zanzotto

(2019) calls for responsible AI with a "human-in-the-

loop" and a clear knowledge life cycle to prevent a

bias of the AI or the chatbot.

4 CONCLUSIONS

Chatbots and conversational AI have the potential to

go beyond the pure simulation or imitation of human

interaction as defined in the introduction. They can

enhance human beings and learners in many possible

ways, individually or in groups, within a classroom or

outside, in business, in education, and daily life. The

human-machine interaction with chatbots in

education promises a variety of pedagogical

advantages and possibilities. Chatbots enable

personalized learner-centred and needs-based

learning, a main success factor for learning, for

student motivation and contribution according to

Chatbot-mediated Learning: Conceptual Framework for the Design of Chatbot Use Cases in Education

213

learning theory and studies. In a collaborative role,

chatbots promise to improve and enhance

collaboration skills, social embedding and team-

reflection. And based on learning analytics and access

to context knowledge including learning paths,

chatbots may make learning visible, improve

metacognitive strategies and foster learner’s

confidence and self-reflection through continuous

monitoring and feedback.

With the presented conceptual framework, we

provide an overview of possibilities how chatbots in

education can be used. The framework might help to

conceptualize a chatbot use case and underlying

pedagogical goals based on a configuration of the

presented dimensions covering the pedagogical,

technological, and content perspectives of an

educational chatbot. Besides the highlighted

pedagogical potential of chatbots in education, we

want to point out limitations regarding our framework

as well as chatbots in education in general. While the

framework presents a high-level understanding and

idea of the configuration, it does not address the

implementation process and its various obstacles that

require utmost attention, e.g. data privacy and

protection, data life cycle, copyrights, integration

issues on institution level, biases, information quality,

dependence on big technology suppliers, ethical and

legal questions and so on.

Future research could consider and focus on these

factors and the implementation phase of concrete use

cases while building upon the conceptual framework

and its underlying concepts and learning theories.

From a pedagogical and interdisciplinary perspective,

it would be interesting to work towards a more

comprehensive evaluation of various success factors

and basic conditions for learning, in addition to

specific evaluations of chatbots in terms of individual

measurable target variables.

REFERENCES

Adamopoulou, E., & Moussiades, L. (2020). Chatbots:

History, technology, and applications. Machine

Learning with Applications, 2(53), 100006.

https://doi.org/10.1016/j.mlwa.2020.100006

Anderson, L. W. (Ed.). (2001). Pearson education. A

taxonomy for learning, teaching, and assessing: A

revision of Bloom's Taxonomy of educational objectives

(Abridged ed.). Longman.

Bandura, A. (1997). Self-efficacy: The exercise of control.

Freemann.

Bodily, R., & Verbert, K. (2017). Review of Research on

Student-Facing Learning Analytics Dashboards and

Educational Recommender Systems. IEEE

Transactions on Learning Technologies, 10(4), 405–

418. https://doi.org/10.1109/TLT.2017.2740172

Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000).

How people learn: Brain, mind, experience, and school.

Expanded edition. National Academy Press.

Cahn, J. (2017). CHATBOT: Architecture, Design, &

Development.

Chen, H., Liu, X., Yin, D., & Tang, J. (2017). A survey on

dialogue systems: Recent advances and new frontiers.

Acm Sigkdd Explorations Newsletter, 19(2), 25–35.

Daugherty, P. R., & Wilson, H. J. (2018). Human +

machine: Reimagining work in the age of AI.

Deci, E. L., & Ryan, R. M. (2012). Self-Determination

Theory. In P. van Lange, A. Kruglanski, & E. Higgins

(Eds.), Handbook of Theories of Social Psychology:

Volume 1 (pp. 416–437). Sage Publications.

https://doi.org/10.4135/9781446249215.n21

Dillenbourg, P. (2016). The Evolution of Research on

Digital Education. International Journal of Artificial

Intelligence in Education, 26(2), 544–560.

https://doi.org/10.1007/s40593-016-0106-z

Dutta, D. (2017). Developing an Intelligent Chat-bot Tool

to Assist High School Students for Learning General

Knowledge Subjects. Georgia Institute of Technology.

https://smartech.gatech.edu/handle/1853/59088

Følstad, A., Araujo, T., & Papadopoulos, S. (2020).

Chatbot Research and Design: Third International

Workshop, CONVERSATIONS 2019, Amsterdam, The

Netherlands, November 19–20, 2019, Revised Selected

Papers (1st ed. 2020). Information Systems and

Applications, incl. Internet/Web, and HCI.

https://doi.org/10.1007/978-3-030-39540-7

Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot

learning partners: Connecting learning experiences,

interest and competence. Computers in Human

Behavior, 93, 279–289. https://doi.org/10.1016/

j.chb.2018.12.023

Garcia Brustenga, G., Fuertes Alpiste, M., & Molas

Castells, N. (2018). Briefing Paper: Chatbots in

Education. Universitat Oberta de Catalunya (UOC).

https://doi.org/10.7238/elc.chatbots.2018

Goel, A., Anderson, T., Belknap, J., Creeden, B., Hancock,

W., Kumble, M., & Wilden, B. (2016). Using Watson

for Constructing Cognitive Assistants. Advances in

Cognitive Systems, 4.

Gunning, D. (2017). Explainable artificial intelligence

(xai). Defense Advanced Research Projects Agency

(DARPA), Nd Web, 2, 2.

Hattie, J., & Yates, G. C. R. (2013). Visible Learning and

the Science of How We Learn. Routledge.

Hobert, S. (2019). How Are You, Chatbot? Evaluating

Chatbots in Educational Settings – Results of a

Literature Review. 10.18420/DELFI2019_289

Huang, J., Lee, K., Kwon, O., & Kim, Y. (2017). A chatbot

for a dialoguq-based second language learning system.

CALL in a Climate of Change: Adapting to Turbulent

Global Conditions, 151.

Ifenthaler, D., & Schumacher, C. (2016). Learning

Analytics im Hochschulkontext. WiSt -

CSEDU 2022 - 14th International Conference on Computer Supported Education

214

Wirtschaftswissenschaftliches Studium, 45(4), 176–

181. https://doi.org/10.15358/0340-1650-2016-4-176

ISTE. (2017). ISTE standards for students.

https://www.iste.org/standards/standards/for-students

Koehler, M., & Mishra, P. (2009). What Is Technological

Pedagogical Content Knowledge? Contemporary

Issues in Technology and Teacher Education, 9, 60–70.

Kumar, R., Rosé, C. P., Wang, Y.-C., Joshi, M., &

Robinson, A. (2007). Tutorial dialogue as adaptive

collaborative learning support. Frontiers in Artificial

Intelligence and Applications, 158, 383.

Leventhal, H., Nerenz, D. R., & Steele, D. F. (1984). Illness

representations and coping with health threats. A

Handbook of Psychology and Health:

Sociopsychological Aspects of Health, 219–252.

Lytras, M., Visvizi, A., Damiani, E., & Mathkour, H.

(2019). The cognitive computing turn in education:

Prospects and application. Computers in Human

Behavior, 92, 446–449.

https://doi.org/10.1016/j.chb.2018.11.011

Meyer von Wolff, R., Nörtemann, J., Hobert, S., &

Schumann, M. (2020). Chatbots for the Information

Acquisition at Universities – A Student’s View on the

Application Area. Chatbot Research and Design - 3rd

International Workshop, 2019, 11970 LNCS.

https://doi.org/10.1007/978-3-030-39540-7_16

Molnar, G., & Szuts, Z. (2018). The Role of Chatbots in

Formal Education. In IEEE 16th International

Symposium on Intelligent Systems and Informatics

(SISY) (pp. 197–202). IEEE.

https://doi.org/10.1109/SISY.2018.8524609

Satow, L. (2019). Lernen mit Chatbots und digitalen

Assistenten. In A. Hohenstein & K. Wilbers (Eds.),

Handbuch E-Learning. Wolters Kluwer.

Seufert, S., Guggemos, J., & Sonderegger, S. (2021).

Soziale Roboter im Bildungsbereich. In O. Bendel

(Ed.), Soziale Roboter (Vol. 22, pp. 475–494). Springer

Fachmedien Wiesbaden. https://doi.org/10.1007/978-

3-658-31114-8_25

Smutny, P., & Schreiberova, P. (2020). Chatbots for

learning: A review of educational chatbots for the

Facebook Messenger. Computers & Education, 151,

103862.

https://doi.org/10.1016/j.compedu.2020.103862

Spierling, U., & Luderschmidt, J. (2018). Chatbots und

mediengestützte Konversation. In C. Kochhan & A.

Moutchnik (Eds.), Media Management (pp. 387–408).

Springer Fachmedien Wiesbaden.

https://doi.org/10.1007/978-3-658-23297-9_22

Turing, A. M. (1950). Computing Machinery and

Intelligence. Mind, LIX(236), 433–460.

https://doi.org/10.1093/mind/LIX.236.433

Walsh, T. (2018). It’s alive: Wie künstliche Intelligenz

unser Leben verändern wird. Edition Körber.

Weizenbaum, J. (1966). ELIZA: A computer program for

the study of natural language communication between

man and machine. Communications of the ACM

9(1966),1."Computational Linguistics" S. 36-45.

Winkler, R., & Söllner, M. (2018). Unleashing the

Potential of Chatbots in Education: A State-Of-The-Art

Analysis. 10.5465/AMBPP.2018.15903abstract

Yu, Z., Xu, Z., Black, A., & Rudnicky, A. (2016). Strategy

and Policy Learning for Non-Task-Oriented

Conversational Systems. Proceedings of the 17th

Annual Meeting of the Special Interest Group on

Discourse and Dialogue.

http://doi.org/10.18653/v1/w16-3649

Zanzotto, F. M. (2019). Human-in-the-loop Artificial

Intelligence. Journal of Artificial Intelligence

Research, 64, 243–252.

Chatbot-mediated Learning: Conceptual Framework for the Design of Chatbot Use Cases in Education

215