Knowledge Controlled Mathematical Coaching

Strategies and Results of a Personalized Blended Learning Approach

Michael Schäfer

Computer Science Institute, University of Applied Sciences Ruhr West, Tannenstraße 43, Bottrop, Germany

Keywords: Mathematical Coaching, Bridging Courses, Personalization, Blended Learning.

Abstract: The mathematical competence of first year students is an important success factor at least for technical

studies. As a significant percentage of students do not have sufficient mathematical skills, universities often

utilise blended learning courses to increase these skills prior to the start of studies. Due to the diversity of

students and their educational backgrounds, individual strategies are needed to achieve the necessary

competence for successfully managing their studies. This paper describes our approach at the University of

Applied Sciences Ruhr West, where we are using personalized blended learning concepts based on the

measurement of individual mathematical competences at the beginning of a coaching process. This is used

to gain a better matching between the individual learner level and the adapted learning concepts. We

combine individual presence learning groups and a personalized e-learning environment. This environment

is adapted based on mathematical skills of each student. It uses individual learning advices, short-term

optical feedback and up to date e-learning material in a Moodle-based LMS (learning management system).

The coaching concept is approved by the results of summative and formative evaluations.

1 INTRODUCTION

In recent years, university professors in technical

studies report decreasing mathematical skills and an

increasing diversity of their educational classes.

Since 2002 the mathematical skills at the

Universities of Applied Science in North Rhine

Westphalia have been tested. The mathematical

competence has been evaluated in a standardized

mathematical test carried out on the first day of their

university education since then.

Ten basic mathematical subjects (e.g. solving

equations, quadratics, powers and logarithms, linear

equation) have to be solved.

The average value of solved questions is shown

in column three of Table 1 (Knospe, 2012).

The already disappointing results of 2002 are

continuously getting poorer.

The approach described here utilizes an

individual adapted mathematical coaching for each

student in order to improve and equalise the

qualifications of all students before the beginning of

their university education.

Table 1: Results of math-examination before the start of

studies.

This paper starts with a section about related

work and continues with a description of the

architectural approach and detailed information

about the implementation, shows results of the

evaluation of the system described and concludes

with an outlook on planned future work.

484

Schäfer M..

Knowledge Controlled Mathematical Coaching - Strategies and Results of a Personalized Blended Learning Approach.

DOI: 10.5220/0004343204840488

In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 484-488

ISBN: 978-989-8565-53-2

Copyright

c

2013 SCITEPRESS (Science and Technology Publications, Lda.)

2 RELATED WORK

In a blended scenario, our approach combines an

online learning course and presence courses that are

adjusted to the results of a mathematical test.

(Schäfer et al., 2012).

The online course is developed due to the

approach of the ARCS-Modell of motivational

design by John Keller (Keller, 2010). It uses

different elements to gain and keep the attention of

the students and to increase the satisfaction, as these

are main factors of learner motivation.

We use several concepts of personalized

feedback to keep motivation and support the

learning outcome. This is similar to concepts of Saul

and Wuttke (Saul and Wuttke, 2011). Regarding the

results of Jarvis and de Freitas (Jarvis and de Freitas,

2009) on the effects of in-game feedback to the

learning transfer improvement, we are using similar

feedback mechanisms. In a first pilot study we

enriched our interface with a humanoid avatar to

improve learning effects as evaluated by Ayad

(Ayad, 2010). The learning design considers aspects

of diversity as shown by Bhattacharya and Hartnett

(Bhattacharya and Hartnett, 2008). To improve the

quality of the coaching concept it is embedded in a

quality improving process based on a modification

of the PDCA cycle (Deming, 1986).

3 ARCHITECTURAL APPROACH

The central idea of our approach is to improve the

matching between the individual knowledge of each

student and our online and offline teaching

strategies. Therefore, each student does a

mathematical test with 48 items out of fifteen topics

as shown in Table 2.

Table 2: Topics of the mathematical test.

The results are stored in a database and used to adapt

the online and presence courses. Further information

is presented in (Schäfer

et al., 2012).

In order to design the presence courses three

clusters of different ability levels are built. Students

with low abilities have courses with a duration of

three weeks starting in small groups with less than

ten participants. Medium-level student courses will

take two weeks while high-level student courses will

last only one week. An overview is presented in

Table 3.

Table 3: Presence learning group arrangement.

New students have to complete the courses

weekly to prevent a separation in different learning

groups corresponding to the ability levels. Students

with low-level abilities are starting in small groups

of 10 participants. After one week students with

medium abilities join these groups up to a maximum

of 25 participants. They are starting again with the

same mathematical subjects, but quicker.

One week later, students with high-level abilities

complete the courses up to 35 participants and the

whole course starts again with the same subjects but

even quicker.

The advantage of this is that the students with

lower abilities had the possibility of repeating the

same subjects several times. The learning velocity of

students with higher abilities is adequately taken into

account. We prevent a separation of students due to

their different skills in the beginning because all

students should recognise an equalized state of

knowledge of the other participants in their course.

The online-learning course starts prior to the

presence courses and it can be attended additionally.

The course is adapted for every student. The results

of the mathematical test are taken into account for a

dynamic generation of personal feedback and for

giving learning advice. The feedback is given with

textual and graphic analysis, symbolic and textual

learning advice. Due to the motivational design by

John Keller, screencasts are used to get attention and

simple mathematical tests with optical feedback are

used to keep attention, to show the learning

enhancement and to support confidence and

satisfaction. Like in serious games the personal aim

is to get as much positive optical feedback as

possible.

The adapted blended learning design as shown

KnowledgeControlledMathematicalCoaching-StrategiesandResultsofaPersonalizedBlendedLearningApproach

485

above is summative and formative evaluated. Due

to this the learning outcome proven and structured

feedback is available to optimise the whole coaching

process.

4 IMPLEMENTATION

The mathematical coaching was developed in the

year 2010 and it was used with 335 students in the

year 2011. After a formative evaluation with the

result of an overall good feedback but a less good

acceptance of the online learning course (Schäfer

et

al.,

2012), the motivational elements were

enhanced. In the year 2012, more than 600 students

took part.

The mathematical test of each student was done

as paper and pencil test. Here, an anonymous

number and the email-address were collected to be

able to send necessary information about the

organisation of the presence and online courses. The

results were stored in an SQL-database being part of

the learning management system (LMS). Moodle

(Moodle Development Team, 2012)

was used as

LMS.

Twenty teachers in three locations have done the

presence courses after a professor of the university

instructed them. Clustering students through

competence and study path did the matching. A

homogenies cluster with a teacher of the same

background/similar degree was built.

An individual mail with the account data to the

LMS, the time and place of the best matching

mathematical courses was generated and distributed.

The online mathematical course combines an all-

embracing amount of learning material with a

pedagogical and motivational concept to improve

the usage of this material and the learning outcome.

The LMS-design is based on the university corporate

design and enhanced with many graphical and

interactive elements. The front page is shown in

Figure 1.

Figure 1: Front page of the mathematical online course.

It gains attention by personally addressing the

student and by using motivating screencasts (Figure

2), which reflects the important of mathematical

knowledge and guides the students through the next

steps.

Figure 2: Screen casts as motivational elements.

They are invited to look at their personal test

score as starting point for their own self-regulated

learning concept.

As shown in Figure 3 the students can see their

own score, the average score of all students and the

expected score the university teachers have. For

simplicity reasons, the scores are clustered by topics

equal to Table 2.

Figure 3: Personal math test result (Blue (first column) –

own score of the student (depending on Moodle user), Red

(second column) – average score of all students, Orange

(third column) – expected score by teachers).

Having had a look at this first feedback, the

students are invited to watch the second screencast

about learning and feedback (Figure 2). Here they

are informed about the feedback symbolism like for

example the thumbs to reflect the results of the

mathematical test in the current subject. Like the

good knowledge of the student with the green thumb

up at the top of Figure 4. After reading these

explanations the students are guided to pass the first

learning element and to do the first mathematical

self-test.

A graphical rating scale gives them feedback on

their success and the students can prove this by

doing a second similar test.

An example is shown in Figure 4.

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

486

Figure 4: Feedback elements in the online course.

To enhance the engagement and the average

length of course-usage an avatar is embedded. This

virtual coach, implemented as 3D humanoid

character, is used to give feedback and learning

advice. Different implementations were evaluated.

On the one hand, a lean design of the automotive

sector as shown in Figure 5 with text-to-speech

translation, which could easily be used for dynamic

interactions (CharAt, Version 1.0) was tested. On

the other hand, pre-recorded but realistic 3D

animated humanoids with synchronized audio from

voice-over artists were tried.

Figure 5: Alex as virtual coach in the online course.

This was topped off with short mathematical

tests at the end of each week in the presence course

and a mathematical test at the end of the coaching

process, which is equivalent to the mathematical test

done before the coaching was started.

5 EVALUATION AND RESULTS

Three different data sources are used for evaluation.

Firstly, it was the log-data of the LMS to prove the

usage of the online mathematical course.

Secondly, the summative evaluation of learning

outcome based on the mathematical tests before and

after the coaching was used.

Thirdly, the formative evaluation one month

after the end of the coaching to evaluate the whole

system was taken into account.

5.1 Log Data

An anonymous ID and an email address were

collected with the mathematical test done in the

beginning. Due to data privacy policies it was not

allowed to use the data from matriculation. With this

data, 613 individual mails with the accounts to the

online course were generated. 32 mails came back

with failure notices. About 50 mails were suspended

to the spam-folders of the recipients by one email-

provider. We suppose that a significant part was not

read, because of the communication shifts from

email to other channels like social media platforms.

In the online course 493 students watched their

personal result as shown in Figure 3. The first

videocast was watched by 191 students (1200 page

views), the first learning element (elementary

calculation) was used by 286 students, the second

from 282 (powers and radicals) students, the fifth

(function) was only used by 149 students. The usage

of selftests have the same tendency from 155

students using the first selftest to 24 students using

the eleventh selftest.

5.2 Summative Evaluation

In the beginning n=613 of all new students took part

in the voluntary mathematical test. This is only a

part of all new students (N=893), because the

matriculation was possible until the first day of

studies. The information about the mathematical

coaching and the account to the online course were

sent to the students one month prior to the start of

studies and the presence course started three weeks

before the studies began.

5.2.1 Results before Coaching

Out of 48 items an average value of AVG=13.70

items were correctly solved with a standard

deviation of SD=8.82.

5.2.2 Results after Coaching

After our coaching the students could pass another

mathematical test with 48 equivalent items (n=132).

The results are based on a paired-samples t-test.

The average value of correctly solved items was

AVG=28.48 with a standard deviation of SD=7.31.

Depending on the kind of eligibility of university

admission, the following differences between the

different groups are to be found:

a) General eligibility with advanced mathematic

course

n=41, AVG=31.90, SD=8.71

KnowledgeControlledMathematicalCoaching-StrategiesandResultsofaPersonalizedBlendedLearningApproach

487

b) General eligibility with basic mathematic course

n=37, AVG=28.51, SD=8.07

c) Subject-linked eligibility

n=50, AVG=26.20, SD=8.69

5.3 Formative Evaluation

A formative assessment is done one month after the

studies start to evaluate the usability, acceptance and

performance of the coaching-system. The survey

consists of 27 items with 12 different dimensions.

The first evaluation in 2011 was only done with a

small part of the participating students (N=49). First

of all we were not sure, if a mathematical test,

before the studies are starting, will be accepted.

With an average value of AVG=5.42, a standard

deviation of SD=.93 and a median of SM=6 the

students seem to accept the test as reasonable.

Visiting the presence learning courses was profitable

for the students with an average value of AVG=5.84,

a standard deviation of SD=1.25 and a median of

SM=6. In a 5-level-Likert scale (Likert, 1932) the

students estimate the influence of the small group-

sizes with an average of AVG=1.7 and a standard

deviation of SD=1.37 (1: very positive, 2:

positive...).

For the question, if using the e-learning platform

was profitable to them, the students estimated with

an average of AVG=3.74, a standard deviation of

SD=1.39 and a median of SM=4. So only a slightly

positive result was measurable. Whereas the visiting

of the presence-learning course in combination with

using the e-learning platform was profitable for the

students with an average of AVG=4.93, a standard

deviation of SD=1.47 and a median of SM=5.

The overall feedback for fitting the demands of

each student, self-observed learning effects and

helpfulness for the first year courses was positive.

We used these results to improve our online-system

as shown before.

6 CONCLUSIONS AND

FURTHER WORK

The knowledge controlled mathematic coaching

concept was successfully implemented. The

summative evaluation shows a significant increase

of the mathematical competences of new students

prior to the beginning of their studies.

We plan to use our results to further enhance the

concept to improve the mathematical competence of

the students and the technical implementation. The

adaption of e-learning material, the personalized

feedback and the arrangement of learning groups

depending on the students’ competence have

positive effects on the improvement of current math

skills of first year students. The enhancement

through avatars seems to be promising.

In conclusion, this paper presented the

implementation of a learner centred adaptable

blended learning concept. The mathematical

coaching significantly improves learning effects.

These effects are controlled by summative and

formative evaluations. Some efforts of future work

are still necessary in order to enhance this concept

and optimise its implementation.

REFERENCES

Ayad, Khaled A. A. (2010). The role of edutainment in e-

learning - an empirical study. PhD Thesis, De

Montfort University.

Bhattacharya M., Hartnett M. (2008). Addressing

Diversity in Design of Online Courses. BCcampus and

Common wealth of Learning.

CharAt (Version 1.0)[Computer Software]. Cologne,

Germany: Charamel GmbH. Retrieved 02/02/2013,

from http://www.charamel.com/.

Deming, W. (1986). Out of the crisis: quality, productivity

and competitive position. Cambridge: Cambridge

University Press.

Jarvis, S.& de Freitas, S. (2009) Evaluation of an

Immersive Learning Programme to Support Triage

Training: In-game Feedback and its effect on Learning

Transfer. In proceedings of IEEE Games and Virtual

Worlds for Serious Applications (VS-GAMES ’09),

pp.117-122.

Keller, J. M. (2010). Motivational Design for Learning

and Performance: The ARCS Model Approach.

Springer US.

Knospe, H. (2012). Mathematik an der Schnittstelle

zwischen Schule und Hochschule Probleme und

Perspektiven. Retrieved 11/06/2012, from

http://www.nt.fh-koeln.de.

Likert, R. (1932). A Technique for the Measurement of

Attitudes. Archives of Psychology 140: 1–55.

Moodle Development Team. (2012). Modular Object

Oriented Dynamic Learning Environment, Open

Source LMS. Retrieved 01/11/2012, from

http://www.moodle.org.

Saul, C. & Wuttke, H.-D. (2011). Feedback

personalization as prerequisite for assessing higher-

order thinking skills. European Journal of Open,

Distance and E-learning, EURODL, pp. 445/1-445/10.

Schäfer M., Jansen M., Stehling T. (2012). Improving

Current Math State of Knowledge for First Year

Students, Proceedings of the 1st International Moodle

Scientific Conference (MoodleSCo2012), 2012

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

488