Use of Multi-touch Gestures for Capturing Solution Steps in

Arithmetic Word Problems

Adewale Adesina

, Roger Stone

, Firat Batmaz

and Ian Jones

Department of Computer Science, Loughborough University, Loughborough, U.K.

Mathematics Education Centre, Loughborough University, Loughborough, U.K.

Keywords: Mathematics, Problem Solving, Multi-touch Interfaces, e-Assessments, e-Learning Tools.

Abstract: Multi-touch interfaces are becoming popular with tablet PCs and other multi-touch surfaces increasingly

used in classrooms. Several studies have focused on the development of learning and collaboration

potentials of these tools. However, assessment and feedback processes are yet to leverage on the new

technologies to capture problem solving steps and strategies. This paper describes a computer aided

assessment prototype tool that uses an innovative approach of multi-touch gestures to capture solution steps

and strategies. It presents a preliminary effort to investigate the capturing of solution steps involving a two-

step arithmetic word problem using the approach. The results suggest that it is possible to perform two step

arithmetic work with multi-touch gestures and simultaneously capture solution processes. The steps

captured provided detailed information on the students’ work which was used to study possible strategies

adopted in solving the problems. This research suggests some practical implications for development of

automated feedback and assessment systems and could serve as a base for future studies on effective

strategies in arithmetic problem solving.

1 INTRODUCTION

Assessment is central to the learning experience

(JISC, 2010). In recent years, there has been an

increasing interest in using technology to enhance

assessment and feedback processes. New

technologies are revolutionizing work, play and

study. The technologies suggest new opportunities to

include touch and physical movement, which can

benefit learning, in contrast to the less direct,

somewhat passive mode of interaction suggested by

a mouse and keyboard. Current research reveals that

the ownerships of technologies such as tablets and

hand held devices among learners are likely to be

widespread (Heinrich, 2011). Other studies have

shown that hand held tablet devices and smart

phones have significant and very positive impact on

learning and motivation of students; leading to

increased capacities to research, communication and

collaboration(Banister, 2010; Gasparini, 2012;

Heinrich, 2011).

Despite the digitally enhanced landscape in

which learning now takes place, assessment and

feedback practices are yet to fully leverage on the

technology to provide innovative solutions to

identified problems. A criticism of some

implementations of computer aided assessment is

that the design sometimes limits creative problem

solving. The most common question type used in

such systems tends to be based on convergent,

selected responses. Some practitioners have argued

that the practice has little pedagogic value beyond

testing surface learning (Hommel et al. 2011).In

solving a two-step arithmetic word problem for

instance, selecting a single best answer among other

options for grading presents some difficulties. First,

only the final answer is compared against the correct

answer, making it difficult to obtain intermediate

results or award partial marks as possible with paper

based assessments. Second, solution paths or

strategies are not explicit. The limited information

on the steps and strategies makes it difficult to give

detailed and personalized feedback on a student’s

work.

This paper discusses a new and innovative

approach to computer-aided assessment that uses

multi-touch gestures to capture solution steps and

strategies. A small pilot study was conducted using

the prototype tool to obtain and examine solution

steps of a two-step arithmetic word problem.

210

Adesina A., Stone R., Batmaz F. and Jones I..

Use of Multi-touch Gestures for Capturing Solution Steps in Arithmetic Word Problems.

DOI: 10.5220/0004415202100215

In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 210-215

ISBN: 978-989-8565-53-2

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

2 APPROACH

Effectively capturing solution steps and strategies

requires a tool that is educationally justified. It must

follow sound pedagogic principles and contribute to

learning, and it should provide an environment that

freely allows creative problem solving without

increasing cognitive load. It should be possible to

capture solution steps without disturbing the user.

Multi-touch interaction is a new technique that

allows users to interact naturally with digital objects

in a physical way, and could help to address the

requirements. The pedagogic advantages of using

gestures have been studied (Drews & Hansen, 2007;

Goldin-Meadow & Beilock, 2010; Segal, 2011) .The

studies show that multi-touch technologies can

benefit cognition and learning (Barsalou,

Niedenthal, Barbey, & Ruppert, 2003), augment

working memory (Goldin-Meadow, 2009) . Also,

the mode of interaction allows for bimanual input

which increase the parallelism of manipulations and

reduce the time of task switching(Jiao, Deng, &

Wang, 2010).

Consider a two-stage arithmetic word problem

that involves three numbers, say 2+5+8. Students are

typically taught to solve the problems in two

separate stages i.e. by adding numbers in pairs.

Fischbein et al. (1985) argued that intuitive models

associate addition with putting together. The first

stage adds 2 and 5; using bimanual multi-touch

interaction makes it possible to simultaneously work

on the two numbers. Although it is possible to use

single touch to interact with the numbers one at a

time, it is rather cumbersome, less intuitive and

requires too many steps. The first step produces an

intermediate result which is used in the next stage.

It is interesting to note the first step has six possible

combinations (2+5, 2+8, 5+2, 5+8, 8+2, and 8+5)

and the second step similarly has six possible correct

combinations of the number pairs (7+8, 10+5, 7+8,

13+2, 5+10, and 2+13). The diversity of solution

paths increases if the other arithmetic operators (-, x,

÷) are required to solve the problems. Capturing the

particular number choices made by the student

during the interactions should provide detailed

feedback on the steps the student has taken to solve

the problem. This feedback provides an opportunity

to examine the strategies adopted in tackling the

problem.

To capture the solution steps without increasing

the cognitive load,(Chandler & Sweller, 1991) the

tool needs to implement a smooth user interface

which allows students to enter the solutions freely

and easily. The interface should present the question

and the solution work areas. For this study, the

problem text and the solution workspaces are placed

together on the same page. This aids the student

memory of the problem context and requirements.

This arrangement is known to have pedagogical

value and has been used in different studies (

Suraweera & Mitrovic 2002; Stone et al. 2009;

Batmaz et al. 2009) .Also, it allows the student to

focus fully and continuously on the task at hand

without having to flip back and forth between pages.

Another advantage is that it facilitates user

interactions between the workspaces with minimal

disruption. The solution space will not provide any

toolbox, options or hints and should allow free form

entry design space.

The method of capturing steps and strategies is

comparable in complexity to that used for design

rationale capture – an area widely studied. Design

rationale has been defined as the reasoning and

argument that leads to the final decision of how the

design intent is achieved (Sims, 1997). A variety of

methods have been used to capture the rationale,

each has its advantage and disadvantages. A method

known as reconstruction method captures the

rationale after the design. This approach does not

interrupt the flow of the design effort but does not

provide accurate or complete rationale capture,

because people usually do not accurately explain

how or why they do things. Another method referred

to in literature as apprentice system Sims 1997),

requires asking the designer questions as the design

action is carried out. This method is time consuming

and frequently interrupts the design effort. A third

approach captures the rationale implicitly. This

approach is used for this work as it does not obstruct

the process and has minimal time overheads.

2.1 The Multi-touch Arithmetic Tool

The prototype tool developed on the iPad is called

the multi-touch arithmetic tool (MAT). The tool

supports questions of different complexities

including all arithmetic operations and provides and

captures solution steps. Figure 1 presents a

description of the tool. It has word problem pane on

which questions are presented to the student and the

solution pane.

The word problem text section presents problems

with numeric values that can be dragged to the

solution area by using simple touch and drag

gestures with one or both hands. The numbers

dragged on the solution pane are referenced to the

numbers on the problem text using techniques

developed by Batmaz and Hinde (2006). The bottom

UseofMulti-touchGesturesforCapturingSolutionStepsinArithmeticWordProblems

211

Figure 1: The Multi-touch Arithmetic tool.

pane is the working pane where the student solves

the problem From the problem pane, the student

chooses the numeric values and drags them to the

solution as illustrated in Figure 1. Two or more

numbers can be moved this way. When this is done

the student simultaneously selects any pair of

numbers by touch holding them for about 3 seconds

(so called long press gesture), this action brings up a

pad containing arithmetic operators from which the

user selects an appropriate operator to solve the

problem (shown in Figure 2).

Figure 2: Performing arithmetic operation with two hands.

For example Figure 2 shows a question involving

the addition of employees, the problem has three

numbers in it which can be carried to the solution

pane. The user selects the two numbers to apply an

arithmetic operator by dragging the numbers

together. Note users can only apply an arithmetic

operator on the number pair they choose. This gives

the opportunity to capture the two numbers the

student is working on. A successful selection of an

arithmetic operator results in a display of a numeric

key pad, through which a calculated result is

inputed.

The result of this intermediate step is fed to the

next stage of the solution process by the same drag

or pan gesture, whiles the touch and hold gesture is

used as above for the arithmetic operation (Figure

3).

Figure 3: Using intermediate results as inputs to other

steps.

The same process is repeated for all numbers and

intermediate results in the problem text until a final

solution is arrived at. Figure 4 shows the feedback of

the solution process.

Figure 4: Feedback on solution steps.

The figure indicates that the first step used for

solving the problem is 7 +53. The result of this step

is 60 – an intermediate result which was used in the

second step. The second step used the result with

the third number i.e. 60 + 6 to obtain a final result of

66. The individual steps and intermediate results can

be assigned marks and graded.

3 PILOT STUDY

The study described in this section was set out to

determine if students can successfully solve the

arithmetic problems using the prototype tool.

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

212

Participants were Loughborough University

students. It is assumed that they will not have

difficulty with arithmetic tasks but are unfamiliar

with the multi-touch approach. Although the

participants are university students, we believe the

findings may be relevant to younger learners as well.

Seventeen students were enrolled for the study. An

introduction session was given to each participant on

sample question to intimate them on how to use the

tool to solve problems. After this, they were asked to

solve three word problems using the techniques

demonstrated. The word problems used are shown

in Table 1.

Table 1: Two Step Arithmetic Word Problems.

Problem Strategy

Q1.William had 7 bottles of

wine. His father gave him 41 more

bottles of wine. His mother gave

him 9 more bottles of wine. How

many bottles of wine did William

have alto

ether?

start

with 41 then

add 9 then

add 7

Q2. Sara has 8 sugar donuts.

She also has 5 plain donuts and 32

jam donuts. How many donuts does

Sara have altogether?

start

with 32 then

add 8 then

add 5

Q3. Jason owned a factory that

employs 53 workers. He hired

another 7 workers. He then hired

another 6 workers. How many

workers are there at the factory

altogether?

start

with 53 then

add 7 then

add 6

It was hypothesized that two main strategies

would be detected from the output results, namely

the place-value strategy in which the student starts

by selecting two numbers that sum to a multiple of

10 in order to reduce computational burden (e.g. 41

+ 9 would be the first number bond in the example

given above), and the ‘as presented’ strategy –

where the order numbers appear in the question is

followed from left to right. We also anticipated that

some students would select numbers arbitrarily.

The numbers were chosen to support the use of

the place-value strategy by students such that in each

problem there is a large (two-digit) number, and a

corresponding small (single-digit) number that sum

to a multiple of 10. In each problem the two-digit

number is presented in a different position: 2nd in

question 1; 3rd in question 2; 1st in question 3. The

particular values were selected so that adding the

single digit numbers was not too easy, i.e. every

single digit addition requires a carry over. The large

numbers were selected such that each question is

most easily answered by starting with the large

number, and then one of the smaller numbers (i.e.

the place-value strategy). Question 3 presents the

numbers in strategic order. This is a control question

to help us work out if any participants consistently

either (i) just chose numbers from left to right or (ii)

just choose numbers arbitrarily.

It can be hypothesized that those with a

conceptual grasp of addition will consistently use the

place-value strategy described above. Those who do

not have a conceptual grasp of addition will go left

to right, select numbers arbitrarily or only make

partial use of the place-value strategy.

4 RESULTS AND DISCUSSION

The present study was designed to determine the

suitability of the multi-touch approach in solving

arithmetic word problems without constraining

problem solvers. The solution steps were captured

for feedback and assessment purposes. The results

obtained from the students showed each step to have

five to six different solution paths.

To assess the usability of the multi-touch

approach, the participants on completing the tasks

were asked to respond on their being able to solve

the problems. The overall response to this question

was very positive, all the participants expressed that

they were able to successfully carry out the tasks.

Analysis of the detailed results generated on the tool

showed that 98% of the participants had correct

answers. Only one participant approached a question

using subtraction rather than addition, and this may

be due to his lack of proper understanding of the

question. To assess how comfortable the participants

were with the solution process, they were asked to

response to a Likert-type question on a six point

scale on how easy it was to use the tool. Over half

(53%), responded that the found the tool moderately

easy to use, 35% found it very easy while the others

(15%) reported using it was sort of easy. While the

study did not set out to test arithmetic ability, the

results suggest that the tool did not prevent the

students from solving the questions and inputting

answers thought to be correct.

Turning now to the question on strategies, the

steps and order in which the participants answered

the problems were all captured. An analysis of the

responses showed different patterns or strategies to

solving the problems can be detected. It was

hypothesized that two major strategies can be

implied from the order the numbers were paired,

(e.g. here we do not discriminate between

UseofMulti-touchGesturesforCapturingSolutionStepsinArithmeticWordProblems

213

participants paired 41 + 7 from those who paired 7 +

41). The strategies output from the tool are

summarized in Table 2. Across all participants just

over half of the questions were solved by starting

with a place-value addition that resulted in a round

number (e.g. 53 + 7). However the use of strategies

varied across the three questions. Fewer than half of

the participants used the place-value strategy for

questions 1 and 2 whereas most participants

appeared to use it for question 3. These between-

question differences were significant, χ

(2, N = 17) =

6.75, p = .034, suggesting participants were more

disposed to using the place-value strategy for

question 3 than they were for questions 1 and 2.

Table 2: Strategies used by the participants to solve the

questions.

STRATEGY Q1 Q2 Q3 TOTALS

place-value 6 7 13 26

other 11 10 4 25

This result is consistent with our hypothesis that

some participants would use the place-value

strategy, and others would use the ‘as presented’

strategy. For questions 1 and 2 those using the place-

value strategy could be discriminated from those

using the ‘as presented’ strategy. However question

3 was deliberately designed such that the place-value

and ‘as presented’ strategies were the same (53 then

7 then 6). Therefore the reason most participants

appeared to use the same strategy in question 3 is

that the place-value and ‘as presented’ strategies

were counted together.

The results suggest that while some participants

were disposed to using the place-value strategy,

overall most participants used the ‘as presented’

strategy on most questions. In light of this finding

we scrutinized the data for evidence of our

hypothesized distinct groupings of participants. The

small sample size (17) and small number of

questions (3) meant this was merely a descriptive

exercise and no generalizable conclusions can be

drawn. Nevertheless, we anticipated a ‘larger’ group

who consistently answered all three questions by

adding the numbers as presented, and a ‘smaller’

group who consistently used the place-value

strategy. To a limited extent this is what we found: 3

of the 17 participants consistently added the

numbers as presented, whereas only 1 consistently

used the place-value strategy. Although these

numbers are small they are encouraging given the

size of the data set, and demonstrate how in

principle the tool might enable the detection of

distinct arithmetic strategies.

5 CONCLUSIONS

This paper has investigated the approach of using

multi-touch gestures to solve two step arithmetic

questions. The pilot study set out to capture solution

steps as the problems were solved and obtain

feedback from the participants on usefulness of the

approach. The results showed that students were

able to freely solve arithmetic problems without

being constrained to limited options or solution

paths. The tool demonstrates that detailed

information on solution steps can be captured

without obstructing a creative problem solving

process. Analysis of captured data suggests that

solution strategies can be detected.

However, the findings are subject to at least two

limitations. First, the study used a convenience

sample size – which was sufficient for descriptive

purposes, but may not suffice to reach generalizable

conclusions on the strategies. Second, university

students were the participants used to acquire

feedback on the approach and to generate multiple

solution paths. While the findings are useful and

applicable to students, they may not be transferable

to children.

Nevertheless, the study suggests several courses

of action: Further experimental investigations on a

larger population involving primary school children

are required to determine if a relationship exists

between strategies and successful problem solving.

The diversity of solution paths is likely to increase

the marking and feedback workloads of teachers if

done manually, a next step will be the study and

development of automated or semi-automated

marking techniques.

REFERENCES

Banister, S., 2010. Integrating the iPod Touch in K–12

Education: Visions and Vices. Computers in the

Schools, 27(2), pp.121–131.

Barsalou, L.W. et al., 2003. Social Embodiment. In

Psychology of Learning and Motivation. Academic

Press, pp. 43–92.

Batmaz, F. & Hinde, Chris J., 2006. A diagram drawing

tool for semi–automatic assessment of conceptual

database diagrams. In 10th CAA International

Computer Assisted Assessment Conference.

Lougborough University, pp. 71–84.

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

214

Batmaz, F., Stone, R. & Hinde, C. J., 2009. Personal

Feedback with Semi-Automated Assessment Tool for

Conceptual Database Model. In 10th Annual

Conference of HEA-ICS. University of Kent,

Canterbury UK, pp. 115–119.

Chandler, P., & Sweller, J. (1991). ). Cognitive Load

Theory and the Format of Instruction Cognition and

Instruction, 8(4),pp.293–332.

Drews, D. & Hansen, A., 2007. Using Resources to

Support Mathematical Thinking: Primary and Early

Years 1st ed., Learning Matters.

Fischbein, E. et al., 1985. The Role of Implicit Models in

Solving Verbal Problems in Multiplication and

Division. Journal for Research in Mathematics

Education, 16(1), pp.3–17.

Gasparini, A.A., 2012. Touch, learn, play - what children

do with an iPad in the classroom. Available at: https://

www.duo.uio.no/bitstream/handle/123456789/9015/G

asparini.pdf?sequence=2 [Accessed January 17, 2013].

Goldin-Meadow, S., 2009. How Gesture Promotes

Learning Throughout Childhood. Child Development

Perspectives, 3(2), pp.106–111.

Goldin-Meadow, S. & Beilock, S.L., 2010. Action’s

Influence on Thought: The Case of Gesture.

Perspectives on Psychological Science, 5(6), pp.664–

674.

Heinrich, P., 2011. The ipad As A Tool For Education A

study of the introduction of iPads at Longfield

Academy, Kent. Available at: http://www.naace.co.uk/

publications/longfieldipadresearch [Accessed

December 20, 2012].

Hommel, B. et al., 2011. Bilingualism and Creativity:

Benefits in Convergent Thinking Come with Losses in

Divergent Thinking. Frontiers in Psychology, 2.

Jiao, X., Deng, H. & Wang, F., 2010. An Investigation of

Two-Handed Manipulation and Related Techniques in

Multi-touch Interaction. In 2010 International

Conference on Machine Vision and Human-Machine

Interface (MVHI). 2010 International Conference on

Machine Vision and Human-Machine Interface

(MVHI). pp. 565 –568.

JISC, 2010. Effective Assessment in a Digital Age: A

guide to technology-enhanced assessment and

feedback.

Kabbash, P., Buxton, W. & Sellen, A., 1994. Two-handed

input in a compound task. In Proceedings of the

SIGCHI Conference on Human Factors in Computing

Systems. CHI ’94. New York, NY, USA: ACM, pp.

417–423.

Segal, A., 2011. Do Gestural Interfaces Promote

Thinking? Embodied Interaction: Congruent Gestures

and Direct Touch Promote Performance in Math.

ProQuest LLC.

Sims, R.C.H., 1997. Interactive learning as an“ emerging”

technology: A reassessment of interactive and

instructional design strategies. Australian Journal of

Educational Technology, 13, pp.68–84.

Stone, R.G., Batmaz, F. & Hinde, Chris J., 2009. Drawing

and marking graph diagrams. ITALICS, 8(2).

Suraweera, P. & Mitrovic, A., 2002. KERMIT: A

constraint-based tutor for database modeling.

Intelligent Tutoring Systems.

UseofMulti-touchGesturesforCapturingSolutionStepsinArithmeticWordProblems

215