Use of Multi-touch Gestures for Capturing Solution Steps in
Arithmetic Word Problems
Adewale Adesina
1
, Roger Stone
1
, Firat Batmaz
1
and Ian Jones
2
1
Department of Computer Science, Loughborough University, Loughborough, U.K.
2
Mathematics Education Centre, Loughborough University, Loughborough, U.K.
Keywords: Mathematics, Problem Solving, Multi-touch Interfaces, e-Assessments, e-Learning Tools.
Abstract: Multi-touch interfaces are becoming popular with tablet PCs and other multi-touch surfaces increasingly
used in classrooms. Several studies have focused on the development of learning and collaboration
potentials of these tools. However, assessment and feedback processes are yet to leverage on the new
technologies to capture problem solving steps and strategies. This paper describes a computer aided
assessment prototype tool that uses an innovative approach of multi-touch gestures to capture solution steps
and strategies. It presents a preliminary effort to investigate the capturing of solution steps involving a two-
step arithmetic word problem using the approach. The results suggest that it is possible to perform two step
arithmetic work with multi-touch gestures and simultaneously capture solution processes. The steps
captured provided detailed information on the students’ work which was used to study possible strategies
adopted in solving the problems. This research suggests some practical implications for development of
automated feedback and assessment systems and could serve as a base for future studies on effective
strategies in arithmetic problem solving.
1 INTRODUCTION
Assessment is central to the learning experience
(JISC, 2010). In recent years, there has been an
increasing interest in using technology to enhance
assessment and feedback processes. New
technologies are revolutionizing work, play and
study. The technologies suggest new opportunities to
include touch and physical movement, which can
benefit learning, in contrast to the less direct,
somewhat passive mode of interaction suggested by
a mouse and keyboard. Current research reveals that
the ownerships of technologies such as tablets and
hand held devices among learners are likely to be
widespread (Heinrich, 2011). Other studies have
shown that hand held tablet devices and smart
phones have significant and very positive impact on
learning and motivation of students; leading to
increased capacities to research, communication and
collaboration(Banister, 2010; Gasparini, 2012;
Heinrich, 2011).
Despite the digitally enhanced landscape in
which learning now takes place, assessment and
feedback practices are yet to fully leverage on the
technology to provide innovative solutions to
identified problems. A criticism of some
implementations of computer aided assessment is
that the design sometimes limits creative problem
solving. The most common question type used in
such systems tends to be based on convergent,
selected responses. Some practitioners have argued
that the practice has little pedagogic value beyond
testing surface learning (Hommel et al. 2011).In
solving a two-step arithmetic word problem for
instance, selecting a single best answer among other
options for grading presents some difficulties. First,
only the final answer is compared against the correct
answer, making it difficult to obtain intermediate
results or award partial marks as possible with paper
based assessments. Second, solution paths or
strategies are not explicit. The limited information
on the steps and strategies makes it difficult to give
detailed and personalized feedback on a student’s
work.
This paper discusses a new and innovative
approach to computer-aided assessment that uses
multi-touch gestures to capture solution steps and
strategies. A small pilot study was conducted using
the prototype tool to obtain and examine solution
steps of a two-step arithmetic word problem.
210
Adesina A., Stone R., Batmaz F. and Jones I..
Use of Multi-touch Gestures for Capturing Solution Steps in Arithmetic Word Problems.
DOI: 10.5220/0004415202100215
In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 210-215
ISBN: 978-989-8565-53-2
Copyright
c
2013 SCITEPRESS (Science and Technology Publications, Lda.)
2 APPROACH
Effectively capturing solution steps and strategies
requires a tool that is educationally justified. It must
follow sound pedagogic principles and contribute to
learning, and it should provide an environment that
freely allows creative problem solving without
increasing cognitive load. It should be possible to
capture solution steps without disturbing the user.
Multi-touch interaction is a new technique that
allows users to interact naturally with digital objects
in a physical way, and could help to address the
requirements. The pedagogic advantages of using
gestures have been studied (Drews & Hansen, 2007;
Goldin-Meadow & Beilock, 2010; Segal, 2011) .The
studies show that multi-touch technologies can
benefit cognition and learning (Barsalou,
Niedenthal, Barbey, & Ruppert, 2003), augment
working memory (Goldin-Meadow, 2009) . Also,
the mode of interaction allows for bimanual input
which increase the parallelism of manipulations and
reduce the time of task switching(Jiao, Deng, &
Wang, 2010).
Consider a two-stage arithmetic word problem
that involves three numbers, say 2+5+8. Students are
typically taught to solve the problems in two
separate stages i.e. by adding numbers in pairs.
Fischbein et al. (1985) argued that intuitive models
associate addition with putting together. The first
stage adds 2 and 5; using bimanual multi-touch
interaction makes it possible to simultaneously work
on the two numbers. Although it is possible to use
single touch to interact with the numbers one at a
time, it is rather cumbersome, less intuitive and
requires too many steps. The first step produces an
intermediate result which is used in the next stage.
It is interesting to note the first step has six possible
combinations (2+5, 2+8, 5+2, 5+8, 8+2, and 8+5)
and the second step similarly has six possible correct
combinations of the number pairs (7+8, 10+5, 7+8,
13+2, 5+10, and 2+13). The diversity of solution
paths increases if the other arithmetic operators (-, x,
÷) are required to solve the problems. Capturing the
particular number choices made by the student
during the interactions should provide detailed
feedback on the steps the student has taken to solve
the problem. This feedback provides an opportunity
to examine the strategies adopted in tackling the
problem.
To capture the solution steps without increasing
the cognitive load,(Chandler & Sweller, 1991) the
tool needs to implement a smooth user interface
which allows students to enter the solutions freely
and easily. The interface should present the question
and the solution work areas. For this study, the
problem text and the solution workspaces are placed
together on the same page. This aids the student
memory of the problem context and requirements.
This arrangement is known to have pedagogical
value and has been used in different studies (
Suraweera & Mitrovic 2002; Stone et al. 2009;
Batmaz et al. 2009) .Also, it allows the student to
focus fully and continuously on the task at hand
without having to flip back and forth between pages.
Another advantage is that it facilitates user
interactions between the workspaces with minimal
disruption. The solution space will not provide any
toolbox, options or hints and should allow free form
entry design space.
The method of capturing steps and strategies is
comparable in complexity to that used for design
rationale capture – an area widely studied. Design
rationale has been defined as the reasoning and
argument that leads to the final decision of how the
design intent is achieved (Sims, 1997). A variety of
methods have been used to capture the rationale,
each has its advantage and disadvantages. A method
known as reconstruction method captures the
rationale after the design. This approach does not
interrupt the flow of the design effort but does not
provide accurate or complete rationale capture,
because people usually do not accurately explain
how or why they do things. Another method referred
to in literature as apprentice system Sims 1997),
requires asking the designer questions as the design
action is carried out. This method is time consuming
and frequently interrupts the design effort. A third
approach captures the rationale implicitly. This
approach is used for this work as it does not obstruct
the process and has minimal time overheads.
2.1 The Multi-touch Arithmetic Tool
The prototype tool developed on the iPad is called
the multi-touch arithmetic tool (MAT). The tool
supports questions of different complexities
including all arithmetic operations and provides and
captures solution steps. Figure 1 presents a
description of the tool. It has word problem pane on
which questions are presented to the student and the
solution pane.
The word problem text section presents problems
with numeric values that can be dragged to the
solution area by using simple touch and drag
gestures with one or both hands. The numbers
dragged on the solution pane are referenced to the
numbers on the problem text using techniques
developed by Batmaz and Hinde (2006). The bottom
UseofMulti-touchGesturesforCapturingSolutionStepsinArithmeticWordProblems
211
Figure 1: The Multi-touch Arithmetic tool.
pane is the working pane where the student solves
the problem From the problem pane, the student
chooses the numeric values and drags them to the
solution as illustrated in Figure 1. Two or more
numbers can be moved this way. When this is done
the student simultaneously selects any pair of
numbers by touch holding them for about 3 seconds
(so called long press gesture), this action brings up a
pad containing arithmetic operators from which the
user selects an appropriate operator to solve the
problem (shown in Figure 2).
Figure 2: Performing arithmetic operation with two hands.
For example Figure 2 shows a question involving
the addition of employees, the problem has three
numbers in it which can be carried to the solution
pane. The user selects the two numbers to apply an
arithmetic operator by dragging the numbers
together. Note users can only apply an arithmetic
operator on the number pair they choose. This gives
the opportunity to capture the two numbers the
student is working on. A successful selection of an
arithmetic operator results in a display of a numeric
key pad, through which a calculated result is
inputed.
The result of this intermediate step is fed to the
next stage of the solution process by the same drag
or pan gesture, whiles the touch and hold gesture is
used as above for the arithmetic operation (Figure
3).
Figure 3: Using intermediate results as inputs to other
steps.
The same process is repeated for all numbers and
intermediate results in the problem text until a final
solution is arrived at. Figure 4 shows the feedback of
the solution process.
Figure 4: Feedback on solution steps.
The figure indicates that the first step used for
solving the problem is 7 +53. The result of this step
is 60 – an intermediate result which was used in the
second step. The second step used the result with
the third number i.e. 60 + 6 to obtain a final result of
66. The individual steps and intermediate results can
be assigned marks and graded.
3 PILOT STUDY
The study described in this section was set out to
determine if students can successfully solve the
arithmetic problems using the prototype tool.
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
212
Participants were Loughborough University
students. It is assumed that they will not have
difficulty with arithmetic tasks but are unfamiliar
with the multi-touch approach. Although the
participants are university students, we believe the
findings may be relevant to younger learners as well.
Seventeen students were enrolled for the study. An
introduction session was given to each participant on
sample question to intimate them on how to use the
tool to solve problems. After this, they were asked to
solve three word problems using the techniques
demonstrated. The word problems used are shown
in Table 1.
Table 1: Two Step Arithmetic Word Problems.
Problem Strategy
Q1.William had 7 bottles of
wine. His father gave him 41 more
bottles of wine. His mother gave
him 9 more bottles of wine. How
many bottles of wine did William
have alto
g
ether?
start
with 41 then
add 9 then
add 7
Q2. Sara has 8 sugar donuts.
She also has 5 plain donuts and 32
jam donuts. How many donuts does
Sara have altogether?
start
with 32 then
add 8 then
add 5
Q3. Jason owned a factory that
employs 53 workers. He hired
another 7 workers. He then hired
another 6 workers. How many
workers are there at the factory
altogether?
start
with 53 then
add 7 then
add 6
It was hypothesized that two main strategies
would be detected from the output results, namely
the place-value strategy in which the student starts
by selecting two numbers that sum to a multiple of
10 in order to reduce computational burden (e.g. 41
+ 9 would be the first number bond in the example
given above), and the ‘as presented’ strategy
where the order numbers appear in the question is
followed from left to right. We also anticipated that
some students would select numbers arbitrarily.
The numbers were chosen to support the use of
the place-value strategy by students such that in each
problem there is a large (two-digit) number, and a
corresponding small (single-digit) number that sum
to a multiple of 10. In each problem the two-digit
number is presented in a different position: 2nd in
question 1; 3rd in question 2; 1st in question 3. The
particular values were selected so that adding the
single digit numbers was not too easy, i.e. every
single digit addition requires a carry over. The large
numbers were selected such that each question is
most easily answered by starting with the large
number, and then one of the smaller numbers (i.e.
the place-value strategy). Question 3 presents the
numbers in strategic order. This is a control question
to help us work out if any participants consistently
either (i) just chose numbers from left to right or (ii)
just choose numbers arbitrarily.
It can be hypothesized that those with a
conceptual grasp of addition will consistently use the
place-value strategy described above. Those who do
not have a conceptual grasp of addition will go left
to right, select numbers arbitrarily or only make
partial use of the place-value strategy.
4 RESULTS AND DISCUSSION
The present study was designed to determine the
suitability of the multi-touch approach in solving
arithmetic word problems without constraining
problem solvers. The solution steps were captured
for feedback and assessment purposes. The results
obtained from the students showed each step to have
five to six different solution paths.
To assess the usability of the multi-touch
approach, the participants on completing the tasks
were asked to respond on their being able to solve
the problems. The overall response to this question
was very positive, all the participants expressed that
they were able to successfully carry out the tasks.
Analysis of the detailed results generated on the tool
showed that 98% of the participants had correct
answers. Only one participant approached a question
using subtraction rather than addition, and this may
be due to his lack of proper understanding of the
question. To assess how comfortable the participants
were with the solution process, they were asked to
response to a Likert-type question on a six point
scale on how easy it was to use the tool. Over half
(53%), responded that the found the tool moderately
easy to use, 35% found it very easy while the others
(15%) reported using it was sort of easy. While the
study did not set out to test arithmetic ability, the
results suggest that the tool did not prevent the
students from solving the questions and inputting
answers thought to be correct.
Turning now to the question on strategies, the
steps and order in which the participants answered
the problems were all captured. An analysis of the
responses showed different patterns or strategies to
solving the problems can be detected. It was
hypothesized that two major strategies can be
implied from the order the numbers were paired,
(e.g. here we do not discriminate between
UseofMulti-touchGesturesforCapturingSolutionStepsinArithmeticWordProblems
213
participants paired 41 + 7 from those who paired 7 +
41). The strategies output from the tool are
summarized in Table 2. Across all participants just
over half of the questions were solved by starting
with a place-value addition that resulted in a round
number (e.g. 53 + 7). However the use of strategies
varied across the three questions. Fewer than half of
the participants used the place-value strategy for
questions 1 and 2 whereas most participants
appeared to use it for question 3. These between-
question differences were significant, χ
2
(2, N = 17) =
6.75, p = .034, suggesting participants were more
disposed to using the place-value strategy for
question 3 than they were for questions 1 and 2.
Table 2: Strategies used by the participants to solve the
questions.
STRATEGY Q1 Q2 Q3 TOTALS
place-value 6 7 13 26
other 11 10 4 25
This result is consistent with our hypothesis that
some participants would use the place-value
strategy, and others would use the ‘as presented’
strategy. For questions 1 and 2 those using the place-
value strategy could be discriminated from those
using the ‘as presented’ strategy. However question
3 was deliberately designed such that the place-value
and ‘as presented’ strategies were the same (53 then
7 then 6). Therefore the reason most participants
appeared to use the same strategy in question 3 is
that the place-value and ‘as presented’ strategies
were counted together.
The results suggest that while some participants
were disposed to using the place-value strategy,
overall most participants used the ‘as presented’
strategy on most questions. In light of this finding
we scrutinized the data for evidence of our
hypothesized distinct groupings of participants. The
small sample size (17) and small number of
questions (3) meant this was merely a descriptive
exercise and no generalizable conclusions can be
drawn. Nevertheless, we anticipated a ‘larger’ group
who consistently answered all three questions by
adding the numbers as presented, and a ‘smaller’
group who consistently used the place-value
strategy. To a limited extent this is what we found: 3
of the 17 participants consistently added the
numbers as presented, whereas only 1 consistently
used the place-value strategy. Although these
numbers are small they are encouraging given the
size of the data set, and demonstrate how in
principle the tool might enable the detection of
distinct arithmetic strategies.
5 CONCLUSIONS
This paper has investigated the approach of using
multi-touch gestures to solve two step arithmetic
questions. The pilot study set out to capture solution
steps as the problems were solved and obtain
feedback from the participants on usefulness of the
approach. The results showed that students were
able to freely solve arithmetic problems without
being constrained to limited options or solution
paths. The tool demonstrates that detailed
information on solution steps can be captured
without obstructing a creative problem solving
process. Analysis of captured data suggests that
solution strategies can be detected.
However, the findings are subject to at least two
limitations. First, the study used a convenience
sample size – which was sufficient for descriptive
purposes, but may not suffice to reach generalizable
conclusions on the strategies. Second, university
students were the participants used to acquire
feedback on the approach and to generate multiple
solution paths. While the findings are useful and
applicable to students, they may not be transferable
to children.
Nevertheless, the study suggests several courses
of action: Further experimental investigations on a
larger population involving primary school children
are required to determine if a relationship exists
between strategies and successful problem solving.
The diversity of solution paths is likely to increase
the marking and feedback workloads of teachers if
done manually, a next step will be the study and
development of automated or semi-automated
marking techniques.
REFERENCES
Banister, S., 2010. Integrating the iPod Touch in K–12
Education: Visions and Vices. Computers in the
Schools, 27(2), pp.121–131.
Barsalou, L.W. et al., 2003. Social Embodiment. In
Psychology of Learning and Motivation. Academic
Press, pp. 43–92.
Batmaz, F. & Hinde, Chris J., 2006. A diagram drawing
tool for semi–automatic assessment of conceptual
database diagrams. In 10th CAA International
Computer Assisted Assessment Conference.
Lougborough University, pp. 71–84.
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
214
Batmaz, F., Stone, R. & Hinde, C. J., 2009. Personal
Feedback with Semi-Automated Assessment Tool for
Conceptual Database Model. In 10th Annual
Conference of HEA-ICS. University of Kent,
Canterbury UK, pp. 115–119.
Chandler, P., & Sweller, J. (1991). ). Cognitive Load
Theory and the Format of Instruction Cognition and
Instruction, 8(4),pp.293–332.
Drews, D. & Hansen, A., 2007. Using Resources to
Support Mathematical Thinking: Primary and Early
Years 1st ed., Learning Matters.
Fischbein, E. et al., 1985. The Role of Implicit Models in
Solving Verbal Problems in Multiplication and
Division. Journal for Research in Mathematics
Education, 16(1), pp.3–17.
Gasparini, A.A., 2012. Touch, learn, play - what children
do with an iPad in the classroom. Available at: https://
www.duo.uio.no/bitstream/handle/123456789/9015/G
asparini.pdf?sequence=2 [Accessed January 17, 2013].
Goldin-Meadow, S., 2009. How Gesture Promotes
Learning Throughout Childhood. Child Development
Perspectives, 3(2), pp.106–111.
Goldin-Meadow, S. & Beilock, S.L., 2010. Action’s
Influence on Thought: The Case of Gesture.
Perspectives on Psychological Science, 5(6), pp.664–
674.
Heinrich, P., 2011. The ipad As A Tool For Education A
study of the introduction of iPads at Longfield
Academy, Kent. Available at: http://www.naace.co.uk/
publications/longfieldipadresearch [Accessed
December 20, 2012].
Hommel, B. et al., 2011. Bilingualism and Creativity:
Benefits in Convergent Thinking Come with Losses in
Divergent Thinking. Frontiers in Psychology, 2.
Jiao, X., Deng, H. & Wang, F., 2010. An Investigation of
Two-Handed Manipulation and Related Techniques in
Multi-touch Interaction. In 2010 International
Conference on Machine Vision and Human-Machine
Interface (MVHI). 2010 International Conference on
Machine Vision and Human-Machine Interface
(MVHI). pp. 565 –568.
JISC, 2010. Effective Assessment in a Digital Age: A
guide to technology-enhanced assessment and
feedback.
Kabbash, P., Buxton, W. & Sellen, A., 1994. Two-handed
input in a compound task. In Proceedings of the
SIGCHI Conference on Human Factors in Computing
Systems. CHI ’94. New York, NY, USA: ACM, pp.
417–423.
Segal, A., 2011. Do Gestural Interfaces Promote
Thinking? Embodied Interaction: Congruent Gestures
and Direct Touch Promote Performance in Math.
ProQuest LLC.
Sims, R.C.H., 1997. Interactive learning as an“ emerging”
technology: A reassessment of interactive and
instructional design strategies. Australian Journal of
Educational Technology, 13, pp.68–84.
Stone, R.G., Batmaz, F. & Hinde, Chris J., 2009. Drawing
and marking graph diagrams. ITALICS, 8(2).
Suraweera, P. & Mitrovic, A., 2002. KERMIT: A
constraint-based tutor for database modeling.
Intelligent Tutoring Systems.
UseofMulti-touchGesturesforCapturingSolutionStepsinArithmeticWordProblems
215