RAPID COMPETENCE DEVELOPMENT IN SERIOUS GAMES

Using Case-based Reasoning and Threshold Concepts

Ioana Hulpuș

1,2

, Manuel Fradinho

, Conor Hayes

Cyntelix, Galway, Ireland

DERI, NUI Galway, Ireland

Leif Hokstad

Learning with ICT, NTNU, Trondheim, Norway

Will Seager, Mick Flanagan

Department of Computer Science, University College London, London, U.K.

Keywords: Rapid competence development, Serious game, Case-based reasoning, Threshold concepts.

Abstract: A major challenge in todays fast pace world is the acquisition of competence in a timely and efficient

manner, whilst keeping the individual highly motivated. This paper presents a novel based on the use of

serious games driven by Case Based Reasoning (CBR) tailored by Threshold Concepts (TC) to present the

learner with the most efficient choice of game scenarios to address their present competence gap. This

allows the learner to maximise their time in competence development. This is current work-in-progress.

1 INTRODUCTION

Serious games for educational purposes have a

number of potential advantages over more

traditional learning methods and on-the-job training.

These include tolerance and encouragement of risk

within a safe environment, thus promoting and

encouraging experimentation instead of passive

learning (Kebritchi and Hirumi, 2008). In addition,

serious games increase motivation, provide ego

gratification, encourage creativity, socialization and

above all are fun. Evidence for their efficacy as

educational tools is growing with a growing number

research studies finding improved rates of learning

and retention for serious games compared with more

traditional learning methods (Druckman and Bjork,

1991; Charles and McAlister, 2004). Therefore,

serious games should must be fun to play but also

effective in supporting learning within the targeted

learning domain. Some argue that many past serious

games have been successful in addressing one of

these objectives but not both: they are either fun to

play but hit-or-miss when it comes to educational

goals and outcomes, or else they effective as

learning tools but stunted as games (Van Eck, 2006).

Sometimes, according to Van Eck (2006), games fail

to achieve either objective and are like offspring

who inherit only the bad features of each parent. An

important aspect of pedagogy is individualization.

Given the variation in learning styles, personal

preferences, well designed games should not create a

“one-size-fits-all” learning environment. In this

context, one question that arises is: “How can we

create game-based learning environments capable of

providing effective learning plans tailored to each

individual learner?”.

To address this question, we start from the

premises that the game-based learning environment

must present the user personalized learning plans

and game scenarios. Moreover, these plans and

scenarios must adapt to the user’s needs as he

performs. We propose a case-based approach to the

generation of learning plans and game scenarios,

which has already been used with serious games

with success. In addition, Case-Based Reasoning

(CBR) has proven to yield good results for the

adaption of on-line tutoring systems. However, the

planning potential of CBR has yet to be exploited in

relation to the creation of learning plans. This being

our main research direction, we take a step further,

374

HulpuÈ

Z I., Fradinho M., Hayes C., Hokstad L., Seager W. and Flanagan M. (2010).

RAPID COMPETENCE DEVELOPMENT IN SERIOUS GAMES - Using Case-based Reasoning and Threshold Concepts.

In Proceedings of the 2nd International Conference on Computer Supported Education, pages 374-379

DOI: 10.5220/0002861503740379

 SciTePress

and address the possibility of integrating the

emerging paradigm of Threshold Concepts.

The paper is structured as follows. In section 2,

we discuss some current approaches that apply CBR

to human learning. This is followed by an overview

of the Threshold Concept paradigm, the benefits it

brings to the design of the learning material. In

section 4, we present our hypothesis on how CBR,

CBP, serious games and threshold concepts can be

used for a learning environment following modern

learning theories. In section 5, we present our

conclusions.

2 HUMAN LEARNING AND CBR

Case-Based Reasoning (CBR) is an artificial

intelligence paradigm that involves reasoning from

prior experiences: it retains a memory of previous

problems and their solutions and solves new

problems by reference to that knowledge. This is the

main difference from rule-based reasoning systems,

that normally rely on general knowledge of a

problem domain, and tend to solve problems from

scratch or from first principles. Usually, the case-

based reasoner is presented with a problem (the

current case). In order to solve it, the reasoner

searches its memory of past cases (the case base) to

find and retrieve cases that most closely match the

current case, by using similarity metrics. When a

retrieved case is not identical to the current case, an

adaptation phase occurs. During this phase, the

retrieved case is modified, taking the differences

into account (Pal and Shiu, 2004). Finally, the cases

are retained in the case base for future use. These

four steps are defined by Aamodt and Plaza(1994) as

Retrieve, Reuse, Revise, and Retain.

Developed from CBR, case-based planning

(CBP) systems address problems that are

represented by goals and have solutions that are

plans. Like traditional case-based reasoners, CBP

systems build new cases out of old ones. Unlike

CBR systems, CBP systems put emphasis on the

prediction of problems: when encountering a new

plan, CBP systems anticipate the problems that can

arise and find alternative plans to avoid the

problems. Plans are indexed by the goals satisfied

and problems avoided (Hammond, 1990).

The idea of using CBR for human learning has

appealed to a number of researchers, partly due to

the roots of CBR in cognitive science which

explains the similarity of CBR to human problem

solving behaviour (Richter and Aamodt, 2005).

There are many examples in the literature of day-to-

day human reasoning and planning that highlight the

important role of previously experienced situations

and of analogy in human problem solving (Schank,

1996; Kolodner et al., 1996).

CBR for human learning purposes has been a

topic of study for a number of years, with significant

developments in the fields of intelligent tutoring

systems and adaptive hypermedia. One of the latest

developments is the ILMDA (Intelligent Learning

Material Delivery Agent), designed by Soh and

Blank (2008). It combines CBR with system meta-

learning for enriching the system with self-

improving capabilities. The learning domain is

computer science for undergraduates. In another

approach, Gomez-Martin et al. (2005) present a

metaphorical simulation of the Java Virtual Machine

to help students learn Java language compilation and

reinforce their understanding of object-oriented

programming concepts. Unlike these two systems,

where the problems have direct mapping to the

correct solution and the targeted domains are well

defined, we are creating a system for use in two very

complex domains: Project Management and

Innovation. In these domains, the problems are

open-ended and the required competences are

complex and difficult to model. Therefore, our

approach is to create an environment capable of

reasoning with very complex, poorly structured

domain knowledge. In addition, we bring

improvements by planning learning based around

longer term goals, rather than one step ahead. It is

worth pointing out that our system is a highly

interactive serious game, instead of text based.

3 THRESHOLD CONCEPTS

The Threshold Concept (TC) Framework focuses on

identifying those aspects of a discipline that are

essential to a grasp of the discipline, that are likely

to be difficult and once overcome will transform the

learner’s view of that discipline. This means the

learner will now begin to think as does a practitioner

of their discipline, e.g., thinks as a manager, thinks

as an innovator. It arose from a study of the teaching

of economics but has now been taken up by

educational researchers and teachers across a wide

range of disciplines (Flanagan, 2009). “Difficulty in

understanding TC may leave the learner in a state of

liminality (Latin limen “threshold”), a suspended

state in which understanding approximates to a kind

of mimicry or lack of authenticity” (Meyer and

Land, 2003). The originators of the framework,

Meyer and Land, characterize the TC as:

(i)Transformative: once a TC is understood, a

significant shift appears in the student’s perception

RAPID COMPETENCE DEVELOPMENT IN SERIOUS GAMES - Using Case-based Reasoning and Threshold Concepts

375

of the subject; (ii) Integrative: once learned, TCs are

likely to bring together and relate different aspects

of the subject that previously did not appear to the

learner; (iii)Irreversible: given their transformative

potential, a TC is also likely to be irreversible,

difficult to unlearn; (iv)Bounded: a TC will probably

delineate a particular conceptual space, serving a

specific and limited purpose; (v)Discursive: Meyer

and Land suggest that the crossing of a threshold

will incorporate an enhanced and extended use of

language; (vi) Troublesome: TCs are likely to be

troublesome for the learner. The framework draws

on Perkins’ discussions of how knowledge may be

troublesome e.g. alien, incoherent or counter-

intuitive (Perkins, 2006). In grasping a TC a student

moves from an apparent ‘common sense’

understanding to an understanding which may

conflict with perceptions that have previously

seemed self-evidently true.

Cousin (2006) suggests some influences that

TCs can have in the design of a university course

curriculum: first, they enable teachers to focus on

what is fundamental to grasp of the taught subject, a

’less is more’ approach to curriculum design; once

identified, the tutor becomes aware of the areas

where students might encounter problems; then, they

might need recursiveness in order to be mastered;

they also require listening from tutor’s side in order

to hear what the students’ misunderstandings and

uncertainties are in order to engage with them

(Cousin, 2006). Cousin characterized in 2009 the TC

framework as a transactional curriculum enquiry

(Cousin, 2009). This would require a partnership

between the discipline’s experts, educational re-

searchers and learners in which curriculum inquiry

and curriculum design are seen as feeding into each

other rather than as sequential activities.

Recently it has been suggested that two

contemporary and powerful conceptual frameworks,

TCs and variation theory share a key pedagogic

principal and share a central common focus (Meyer

et al, 2008) warranting further examination.

Although first used in (Dienes, 1967), variation

theory of learning is now associated with a much

more formalized approach rooted in

phenomenography (Marton and Booth, 1997). It

states that a key feature of learning involves

experiencing that phenomenon in a new light

(Marton and Trigwell, 2000). Marton argues that

“there is no learning without discernment and there

is no discernment without variation”. Therefore, in

order for students to discern the object of learning,

they must experience how they vary. The key

elements that are relevant here may be summarized

as its four patterns of variation: (i)contrast -

experience something else to compare it with,

(ii)generalization - experience varying appearance of

an object, (iii)separation - experience a certain

aspect of something by means of varying it while

other aspects remain invariant and (iv)fusion -

experience several critical aspects simultaneously.

The work of Bernhard’s group (Cartensen and

Bernhard, 2008) on applying variation theory to a

circuit analysis problem in which a TC is embedded

and the study by Flanagan, Taylor and Meyer (2009)

on how a TC in engineering comes into view when

approached from two very different engineering

contexts, strengthen Meyer and colleagues’

suggestion for further examination. Problem-based

learning has also been suggested in (Biz/ed, 2009)

for facilitating a learner’s traverse across the liminal

space. Other recent studies of Meyer and colleagues

(Meyer et al., 2009) show how meta-learning can

help at overcoming TCs and its importance in

identifying transformation. To sum up, all these

studies show positive results over the improvement

of the learning process by integrating TCs. In this

context, we consider that TCs are indispensable for

an efficient, beyond the current state-of-the-art,

learning environment.

4 A NEW APPROACH

To the best of our knowledge, there has been no

previous work addressing how the TCs can be

incorporated into serious game design. The

consideration of the suggested roles that TCs might

play in the design of university curriculum augurs

well for a neat transfer to game-based curriculum

design. Nevertheless, the “transactional curriculum

enquiry” aspect does not lend itself quite so readily

to the serious game environment. Still, it might well

be accommodated by a serious game envisaged as a

component of blended learning where mentors

facilitate a learner’s traverse across the liminal space

encountered on meeting a TC.

The idea of using a blended learning mixing the

above mentioned learning theories resonates with a

serious game using case-based reasoning. First of all

because the design of a serious game must be based

on established instructional strategies and learning

theories (Kebritchi and Hirumi, 2008; Charles and

McAlister, 2004; Van Eck, 2006) and problem-

based approaches already proved a high potential in

game-based learning. The synthesis of CBR and

problem-based learning has been the object of

several studies suggesting a fruitful fusion. The

adaptive nature of CBR lends itself for including

ideas of variation theory of learning into serious

games and CBR has also been studied in relation

CSEDU 2010 - 2nd International Conference on Computer Supported Education

376

with Meta-Learning (Soh and Blank, 2008) showing

that a detailed analysis and adaption of the learning

process can be used to improve students’ results.

A “by-product” of CBR is the case-base which

brings together all the experiences created by

learners using the system. This enables the

environment to integrate case-based learning (CBL).

CBL allows the students to view how others act,

analyze and compare with their own actions and has

already been successfully used in serious games.

Moreover, the case-base can be analyzed in order to

identify learner models enabling the adaption of

plans for each such model. Another analysis

direction would be to determine if action patterns

exist, which might lead to the identification of TCs.

By analyzing the dynamics of the cases, we expect it

is also possible to identify learners’ passages

through the liminal space. Considering Davies’ and

Mangan’s claim (2006) that TCs cannot be isolated

from the social background of the learning process,

we can analyze how TCs are cultural dependent and

if their grasping difficulty differs from one learning

community to another.

To the best of our knowledge, there exists no

previous work that combines so many learning

strategies, and that utilizes the established case-base

from such a wide variety of angles.

In Figure 1, we illustrate how CBR can be

incorporated into the learning process within a

game-based learning environment. At the start of the

process, the learner decides to achieve more

competences. A case for the case-based planner is

derived from the plan goal (targeted competences),

by the set of possible intermediate goals

(competence gap), and the plan preconditions (the

learner model and his current competences).

Drawing on this data, the system uses case-based

planning to generate personalized plans for the

learner. From a list of recommended games, the

learner chooses the first game to play. As he or she

plays, an experience is generated and added to his or

her trail.

Figure 1: Proposed case-based learning process.

Depending on the learner’s performance, the sys-tem

decides if the intermediate competences have been

achieved. If the learner has failed to achieve them,

the case-based planner identifies the situation as a

failure and tries to recover in two ways: i) the

planner anticipated the problem and will have

already assigned a recovery plan for a particular

story. If this is the case, the planner will choose the

recovery plan with highest eligibility value; ii)

otherwise the planner will undergo a CBR process to

recommend other stories to the learner in order to

bring him or her to the required standard in relation

to the intermediate competences. This is similar to

the process suggested by variation theory in which

stories associated with TCs are adapted to help the

learner master the concept. When all the goals of the

plan have been achieved, the trail is saved and

becomes part of the case base.

4.1 The Case Base

In a CBR system, a case is a pair problem-solution.

In our system, the problem is represented by the

goal, preconditions and learner model, as shown in

Figure 1. Still depending on the solution, we have

two kinds of cases: story cases, where the solution is

a story, and plan cases where the solution is a plan.

A plan is an ordered sequence of stories.

Experiences are instances of stories generated each

time a learner plays a story, whereas trails are

instances of plans, therefore sequences of

experiences. Experiences and trails are used to

evaluate the stories and plans respectively.

On the basis of these definitions, we can

formalize the case knowledge of the system as

containing a set of knowledge assets with a story at

the core. Each story holds references to the

experiences it has seeded. The stories are

interconnected into plans, which are associated with

a set of trails that link together experiences. These

knowledge assets have associated social data created

by the community, such as feedback, ranking, peer

assessment, tags, etc. This leads to a very big search

space and consequently to the challenge of indexing

knowledge chunks.

4.2 Generating the Plans

The plan generation described above uses a case-

based planning approach based on 4 phases.

Plan Retrieve. Starting with the goals and

preconditions, the planner searches the case base to

find plans with similar descriptions, which yielded

good results for the learner. In order to do this, the

system must consider different types of knowledge

RAPID COMPETENCE DEVELOPMENT IN SERIOUS GAMES - Using Case-based Reasoning and Threshold Concepts

377

and reasoning methods such as similarity metrics,

utility metrics, statistical reasoning and collective

filtering. The exact combination of reasoning

methods is still an open issue. Another challenge is

to decide how to weight and then combine all the

obtained values. We will consider shifting this

responsibility to the system itself by making it

capable of analyzing the outcome using its own

reasoning and adapting the measures accordingly.

However we proceed, one principle that we will

follow in our work is for the system to recommend

and orient the learner, while allowing the learner to

choose from the list of recommended options. If the

goal competences are related to TCs, the planner

will retrieve the plans and stories which were most

successful in facilitating the learners in surpassing

the threshold. Another focus of research related to

this phase concerns the situation where student are

new to the system. In this situation, the system will

not yet hold enough information to be able to assign

a learner model to the student. In this context, a

conversational CBR (CCBR) approach might be

used. A CCBR system is used when the problem is

not completely known and, therefore, the traditional

retriever has no data to match the cases to. The

system starts a conversation with the user, asking

him questions which discriminate between learner

models by traversing a decision tree. As the learner

model is drawn out from this conversation, and the

other problem data are know, the system selects the

suitable cases. An even more attractive direction

would be to adapt CCBR so that, instead of using

conversations to figure out the learner model,

learners are given stories to play, where the stories

are chosen in such a way that the user’s actions lead

the reasoner along the same discriminative decision

tree.

Plan Reuse and Revise.The differences between the

goals of retrieved plans and the goals of the current

learner are identified and used to adapt the plan. If

the goal competences are not similar, the

competences to be removed are identified and the

associated stories are removed from the plan. If the

current targeted competences usually entail the

mastery of some TCs, the plan is adapted so that it

targets these TCs. The obtained plan and stories are

then analyzed using domain knowledge to make sure

that they are coherent, and revised if needed.

Plan Retain. The plan and its trail are saved in a

temporary storage after it has been played by the

learner. Then, periodically these plans and trials are

analyzed and filtered. For the stories which failed

(eg.: the learner did not achieve the related

competences), the planner updates its fail

expectation, and saves the recovery plan which

worked. The recovery plan is represented by the

stories the learner played until they achieved those

competences. At this stage, if the plan is a new one,

it is assigned a utility and eligibility value. If the

plan is a reused one, these values are updated. When

a contingency story has a better eligibility value than

the story in the original plan, it replaces the story in

the plan. An important challenge here is to filter out

the plans and stories which are not considered

relevant for future use.

4.3 When the Learner Gets Stuck

Besides plan generation, we use a case-based

reasoner to recommend stories which might help the

learner get over the stages where he or she gets stuck

in the learning process. When the learner fails to

achieve the supposed intermediate goals, the planner

detects a fail. This failure might be interpreted by

the planner as either an expectation failure or plan

failure. A learner might get stuck in a game by not

making any relevant progress, which leads to

frustration. In this case, the case-based reasoner

suggests targeted stories or story episodes, starting

from one which poses problems to the learner, but

adapted based on the variation patterns from

variation theory. This adaptation must be made in

relation to a TC, if one has been identified to be

involved in the difficulty. The system would also

recommend to the learner that they watch similar

experiences to see how other learners handled the

situation, and actually provide the option of allowing

the learner to replay the games. In addition, the

system might show the learner graphs and statistics

on their performance, their learning patterns, and if

possible, suggest enhancements of his learning style.

In this way, learners would have the chance to

analyze their overall progress and how it was

achieved, and thereby have facilitated the process of

meta-learning. Associated with this, the system can

improve its reasoning as it is being used by

analyzing which of its suggestions were most

beneficial, giving these a bigger weight in the future.

5 CONCLUSIONS

The paper outlines the work-in-progress concerning

the research of rapid competence development in

shorter time-to-competence. The approach is based

on the use of serious games, where the learner’s

plans are composed of games determined by a

reasoning process that combines CBR with CBP,

shaped by TC. Initial concepts have been

experimented resulting in the proposed framework

CSEDU 2010 - 2nd International Conference on Computer Supported Education

378

with the next phase involving the integration with

the serious game supporting competence

development in both project management and

innovation. Further research is being carried out to

validate the approach and evaluate the effectiveness

of learning, including the role and impact of TCs.

This presents the challenge of determining which

plans and stories have previously been most

effective in helping learners to grasp the TCs.

ACKNOWLEDGEMENTS

This work was partly supported by the TARGET project

under contract number FP7-231717 within the Seventh

Framework Program, and by the ‘Líon II’ project funded

by SFI under Grant SFI/08/CE/I1380. The authors also

wish to thank Marcel Karnstedt for the many fruitful

discussions.

REFERENCES

Aamodt, A. and Plaza, E. (1994). Case-based reasoning:

Foundational issues, methodological variations and

system approaches. AI Communications, 7(1):39–59.

Biz/ed (2009). Threshold concepts: Problem-focused

learning. a guide for teachers and lecturers,.

http://www.bized.co.uk/educators/he/threshold/pflintro

.htm.

Cartensen, A. and Bernhard, J.(2008), Threshold Concepts

and Keys to the Portal of Understanding: Some

Examples from Electrical Engineering, In Threshold

Concepts within the Disciplines, Land, R., Meyer,

J.H.F. and Smith, J., (eds), Sense Publishers,

Rotterdam, pp 143−154

Charles, D. and McAlister, M. (2004). Integrating Ideas

About Invisible Playgrounds from Play Theory into

Online Educational Digital Games, volume 3166,

pages 598–601. Springer Berlin / Heidelberg.

Cousin, G. (2006). An introduction to threshold concepts.

Planet, (17).

Cousin, G. (2009). Researching Learning in Higher

Education: An Introduction to Contemporary Methods

and Approaches, pages 183–199. Routledge.

Davies, P. and Mangan, J. (2006). Trajectories of students’

learning: threshold concepts and subject learning

careers. Annual Conference of the Society for

Research in Higher Education, University of Brighton,

13th-15th December 2006.

Davies, P. and Mangan, J. (2007). Embedding Threshold

Concepts: from theory to pedagogical principles to

learning activities. Sense Publishers.

Dienes, Z. P. (1967). Building up mathematics. Education

Ltd.

Druckman, D. and Bjork, R. A. (1991). In the Mind’s Eye:

Enhancing Human Performance. The National

Academies Press.

Flanagan, M. (2009). Threshold concepts: Undergraduate

teaching, postgraduate training and professional

development: A short introduction and reference list.

http://www.ee.ucl.ac.uk/ mflanaga/thresholds.html.

Flanagan, M. T., Taylor, P., and Meyer, J. (2009).

Compounded thresholds in electrical engineering, in

threshold concepts and transformational learning. in

Threshold Concepts and Transformational Learning,

Land, R., Meyer, J.H.F. and Baillie, C., (eds), Sense

Publishers, Rotterdam, in press.

Gomez-Martin, P. P., Gomez-Martin, M. A., Diaz-Agudo,

B., and Gonzalez-Calero, P. A. (2005). Opportunities

for cbr in learning by doing. ICCBR, pages 267–281.

Hammond, K. J. (1990). Case-based planning: A

framework for planning from experience. Cognitive

Science: A Multidisciplinary Journal, 14(3):385–443.

Kebritchi, M. and Hirumi, A. c. (2008). Examining the

pedagogical foundations of modern educational

computer games. Comput. Educ., 51(4):1729–1743.

Kolodner, J. l., Hmelo, C. E., and Narayanan, N. H.

(1996). Problem-based learning meets case-based

reasoning. In In Proceedings of the Second

International Confer-ence of the Learning Sciences.

Marton, F. and Booth, S. (1997). Learning and Awareness.

Lawerence Erlbaum Associates, New Jersey.

Marton, F. and Trigwell, K. (2000). Variatio est mater stu-

diorum. Higher Education Research and Development,

19(3):381–395.

Meyer, J.H.F., Land, R. and Davies, P. (2008), Threshold

concepts and troublesome knowledge (4): Issues of

variation and variability, in: Land, R., Meyer, J.H.F.

and Smith, J., (eds), Threshold Concepts within the

Disciplines. Sense Publishers, Rotterdam, pp 59−74

Meyer, J., Ward, S., and Latreille, P. (2009). Threshold

concepts and metalearning capacity. International

Review of Economics Education, 8(1):132–154.

Meyer, J. H. F. and Land, R. (2003). Threshold concepts

and troublesome knowledge (1): linkages to ways of

thinking and practising within the disciplines.

Improving Student Learning - Ten Years On.

Pal, S. K. and Shiu, S. C. (2004). Foundations of Soft

Case-Based Reasoning. John Wiley and Sons, Inc.,

Hobo-ken, New Jersey.

Perkins, D. (2006). Constructivism and troublesome

knowledge in Overcoming Barriers to Student

Understanding: threshold concepts and troublesome

knowledge. Routledge-Taylor

Richter, M. M. and Aamodt, A. (2005). Case-based

reasoning foundations. Knowledge Eng. Review,

20(3):203– 207.

Schank, R. C. (1996). Goal based scenario: Case-based

reasoning meets learning by doing. Case-Based

Reasoning: Experiences, Lessons and Future

Directions, pages 295–347.

Schank, R. C. and P., A. R. (1977). Scripts, plans, goals

and understanding: an inquiry into human knowledge

structures. Hilsdale, NJ: Erlbaum.

Soh, L. and Blank, T. (2008). Integrating case-based

reasoning and meta-learning for a self-improving ITS.

Journal of Artificial Intelligence in Education,

(18):27–58.

Van Eck, R. (2006). Digital game-based learning: It’s not

just the digital natives who are restless, EDUCAUSE

Review.

RAPID COMPETENCE DEVELOPMENT IN SERIOUS GAMES - Using Case-based Reasoning and Threshold Concepts

379