Development of a Skill Learning System using Sensors in a Smart

Phone for Vocational Education

Akinobu Ando

, Toshihiro Takaku

, Shota Itagaki

, Takashi Torii

Hidetoshi Takeno

and Darold Davis

Miyagi University of Education, 149 Aramaki-Aoba, Aoba-ku, Sendai, Japan

Sugiyama Jogakuen University, 17-3 Motomachi, Hoshigaoka, Chikusa-ku, Nagoya, Japan

Hiroshima Institute of Technology, 2-1-1 Miyake,Saeki-ku, Hiroshima, Japan

Replicant AD, LLC, 1145 Walnut Street, CA 94707-2616, Berkeley, U.S.A.

Keywords: Vocational Education, Skill Learning, Smart Phone, Feedback Own Movement.

Abstract: Generally speaking, learning or teaching requires time, particularly with physical movement. In this paper,

we demonstrate an efficient way for students to learn skills by using sensors in a smartphone. Every smart

phone has some type of sensors e.g. acceleration sensor, gyro sensor and so on. Our “Skill Self Learning

System (SSLS)” can give feedback, results and evaluation of a student’s actual movement to his/her self

immediately on the smart phone, which he/she is using. In this research we used SSLS for learning how to

saw and plane. As a result of running simple tests, a group using SSLS could improve their skills as well as

a group, which a teacher taught in person. According to the results, if students have learned the skills before

an actual class, a teacher can teach skills in detail and support them interactively. Our method is beneficial

for enhancing traditional vocational learning as well as for distance skill education, intelligent skill learning

and teaching system.

1 INTRODUCTION

For efficient learning, it is considerably important

for learners to have knowledge about the contents

before their class. Flipped-learning is one typical

approach. The flipped learning/teaching method is

defined as follows, according to Wikipedia (2011):

“Flip teaching is a form of blended learning which

encompasses any use of Internet technology to

leverage the learning in a classroom, so the teacher

can spend more time interacting with students instead

of lecturing”.

Moreover, Berrett (2012) mentions the benefit of the

learning style. As a background, the important point

made is that time is too short to learn only in face to

face class environment. Needless to say, this

problem occurs in skill learning as well. It is a

particularly serious problem for Japanese junior high

school students who do not have real experience

(Ando and Abiko, 2003). In this paper, we focus on

vocational learning, and the subject is technology-

education in Japan. In Technology-Education,

students learn to “make things” such as the craft arts

(woods, plastics and metals), electricity, metal craft,

materials, changing energy and Information

Technology. In the Japanese education system, the

subject is allocated only one class per week and is

found only in Junior high school education

curriculum. Therefore, not only have almost all

students never tried to use craft tools e.g. a saw and

a plane, but also they don’t have enough time to

learn its skill. In spite of the fact that these are basic

skills in Technology Education, a lot of students

have never been good at making things (Yamamoto

et al., 2007). Obviously, the unique circumstances

around students are changing. Competently, most

students don’t feel the need to make things in their

life. However the experience of making useful

products according to their own design affects not

only national power, but also desired character

formation (Ando et al., 2011). It is commonly

known that learning skills to use tools requires the

necessary equipment and right conditions.

Unfortunately, such tools, i.e. a saw and plane, are

not readily available in students' home. Moreover,

683

Ando A., Takaku T., Itagaki S., Torii T., Takeno H. and Davis D..

Development of a Skill Learning System using Sensors in a Smart Phone for Vocational Education.

DOI: 10.5220/0004349406830687

In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 683-687

ISBN: 978-989-8565-53-2

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

most parents themselves aren't good at these skills,

therefore the students cannot understand if their

posture and movement are correct or not. MEXT

(Ministry of Education, Culture, Sports, Science and

Technology in Japan) (2011) says teachers should

make students into well-learned citizens, however a

concrete teaching program hasn't been developed to

meet such a need.

On the other hand, there exists research

regarding the usage of sensors to understand a user’s

state or action. For instance, Hattanda et al. (2008)

has tried to develop new functions of a cell-phone in

which the screen changes automatically in response

to adapting to user actions. In this work, it required

acceleration sensor hardware connected outside of

the cell-phone. At around this time, sensor devices

had been the focus of mobile devices. In the

educational field, Kashiwagi et al. (2010) began to

research the use of a few acceleration sensors and

slant sensors attached to a students’ head, both sides

of wrists, both sides of ankles and waist. These

sensors were hardware which could transmit data to

a PC. Using this system, an observer could know

how students behaved from a distant place.

The originality of our work is in using a

smartphone instead of special devices. All

smartphones are equipped with acceleration sensors,

slant sensors and gyro sensors. Though it is possible

that special sensors may be far more precise than

sensors in a smartphone, we haven’t found practical

reports for the investigation of a user’s condition by

using a smartphone. Moreover, we haven’t found

any previous research that exists regarding an

application of a smartphone to learn or improve a

students’ skill. Our research question is sensing data

from sensors in a smartphone that can enhance the

students’ skills. As we described, there are serious

problems in vocational education. If this approach is

developed, students’ are able to learn, practice and

improve their skill without a teacher and special

classroom. Former models of smartphones are less

expensive and even an obsolete model can be used

in this approach.

In this research, we aimed to develop a "skill

self-learning system (SSLS)" and evaluate the

approach for beginner subjects to improve their skill.

We described a beginner’s typical mistakes of

sawing and planning. Then, we suggested the

necessity of self-skill-learning and advantages of

using a smartphone. Finally, the results of a simple

experimentation by using our developed system

were discussed.

2 TYPICAL FAILURE

SITUATION FOR A BEGINNER

Sometimes, typical failure situations appear while

beginner students practice their own skill without the

aid of a teacher. Situations may be errors such as the

wrong strength or incorrect movement. Japanese

official textbook (2012) for technology and

vocational education says

“While sawing, set the

blade as shown, like a straight line” and the other

textbook says “to cut straight, place the blade

underneath, between the eyes”. In addition, the

textbook says “the angle between a wood piece and

a blade should be from fifteen to thirteen degrees”.

When sawing, a blade should be moved forward and

backward repeatedly while keeping the blade

vertical. According to practical experience, a

beginner tends to fail to correctly do the three

patterns as follows; first, a worker cannot set and

keep the blade at the proper angle. The second,

while moving the blade, it leans to the side and it

isn’t kept vertically straight. The last, the blade is

moved irregularly. Unfortunately, a beginner cannot

be made aware of these typical failures unless

someone tells them.

In the case of using a “plane” to flatten the

surface of wood, it is difficult for a beginner to slide

it at the proper speed. Moreover, at the beginning of

sawing, the peculiar movement, which a plane is

moved marginally to a backward direction, appears

in the expert’s movement. It is also hard for a

beginner to understand how to coordinate his/her

arms and his/her waist.

3 OUTLINE OF THE SYSTEM

3.1 Basic Function and Contrivance

Our original SSLS method is to use "smart-phone"

instead of special sensors and analysing system.

Every smart phone has some important features:

LCD wide screen, gyro sensor, acceleration sensor,

data storage and connection to the Internet. Our

basic design allows students to attach a smart phone

on a tool and assess their own skill without an expert

by displaying feedback results. The SSLS works on

devices installed with Android OS version 2.3 or

greater. The SSLS is able to record and calculate

data every ten micro seconds. This data represents

the number of forward and backward motions, speed

of moving, maximum speed, elapsed time, change of

slant angle while moving, and the change of blur

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

684

while moving.

In the case of sawing, the acceleration of the y

axis and z axis is calculated using the three axis

acceleration sensor of the smart phone, in order to

count the number of times the blade moves forward

and backwards. Figure 1 shows the actual results of

an expert worker’s sawing.

Figure 1: Change acceleration of an expert’s sawing.

In this research, the threshold value for counting

was fixed at 15(m/s

). To record the slant angle of a

blade, it uses values taken from the slanted sensors.

After a few practice sessions, the application

displays the initial results as feedback information to

the student. By these results, a student can improve

his/her own movement and skill (Figure 2). In the

case of planning, the application shows the change

of acceleration as a line graph comparing them with

the results of an expert’s movement, and feedback

message regarding motion speed (Figure 3).

Figure 2: Result and advice for improving sawing.

Figure 3: Result and advice for improving planning.

3.2 How to Set up and Practice

In the case of sawing, a smart phone, which was

installed with our application, is put on a bar of a

saw with an attachment (Figure 4, left). To minimize

the weight of the whole saw, the attachment is made

of foaming polystyrene. In the case of planning, the

smart phone is put on a wooden piece, like the body

of plane (Figure 4, right). In both cases, hook-and-

loop fasteners secure the smart phone to the tool.

Figure 5 shows a practice motion and Figure 6

shows actual scenes of sawing and planning.

Figure 4: A smart phone put on a bar of a covered saw

(left) and a smart phone put on a wood piece like a plane

(right).

To begin a practice session, tap to run SSLS

application for sawing. Before sawing, the student

must adjust the proper angle between the blade and

wood. Then tapping the start button, SSLS begins to

record and analyse the acceleration of the smart

phone. During practice, if SSLS detects improper or

irregular motion, the smart phone screen changes

colour from blue to red and beeps to indicate

incorrect movement. After sawing, SSLS shows a

summary of the practice results. However, in order

to practice planning, a student has to watch a

recorded example of correct planning posture

beforehand. To record proper examples, tap the

“record button” on the menu screen. In this record

mode, SSLS can record whole movements of

planning. Figure 7 and Figure 8 show each algorithm

of practice.

Figure 5: Actual scene of sawing with a smart phone.

00:00 03:20 06:40 10:00 13:20 16:40 20:00 23:20 26:40

経過時間(s)

合成加速度(m/s2)

Total score of the practice session

Total number of movements

Total number of incorrect

Total score of the practice session

Appropriate speed of movement

Evaluation of practice

Max and Min speed

Elapse time of movement

Total evaluation of motion blurs

Red line: ideal changing of acceleration

Blue line: result of practice session

Time

Acceleration (m/s

)

DevelopmentofaSkillLearningSystemusingSensorsinaSmartPhoneforVocationalEducation

685

Figure 6: Actual scene of planning with a smart phone.

Figure 7: Algorithm in sawing practice.

Figure 8: Algorithm in planning practice.

4 EXPERIMENTAL PROCEDURE

To evaluate our SSLS approach for improving

students sawing skill experiment, we conducted test

with seven university freshman subjects who were

beginners, and we conducted test with six subjects

who were high school students.

When sawing, the most important skill is to cut

the wood straight and vertical. So, we compared

changes of the blade’s slant by using the SSLS and

by a teacher’s oral method (Oral teaching group).

All subjects used a covered saw with the smart

phone and practiced moving it forward and

backward repeatedly twenty times. In the smart

phone group, subjects checked their own results and

feedback information on the smart phone during

every trial. In the other group, a teacher instructed

subjects of the oral teaching group during every

session. Subjects of both groups tried this repeatedly

five times. We compared the average acceleration of

the sideward motion of the first result with average

acceleration of the fifth result.

Next, in the case of planning, we compared the

first and fifth sessions’ acceleration values change of

the blur made by the body of the moving plane. In

this case, all subjects were in separate SSLS groups;

Oral teaching group as well as sawing. Both groups

consisted of three subjects.

5 RESULT OF PRACTICE TEST

Figure 9 shows the result of sawing test. In first

instance, average acceleration of the blade was 0.61

(m/s

) (SD=0.34) in SSLS group and 0.50 (m/s

)

(SD=0.28) in the oral teaching group, and at the last

time, it was 0.56 (m/s

) (SD=0.31). This result

shows that only SSLS had significant differences

(603)

= 2.475, p< 0.05). It means that subjects

improved by using SSLS. The first time however,

there were already apparent differences of skill

between the SSLS group and the textbook group.

Subjects of the oral teaching group were already

good at sawing. Therefore, we could not conclude

that using SSLS was better than the oral teaching

methods according to only this result.

Likewise, Figure 10 shows the result of planning

test. It was found that both groups improved in blur

of planning (p< 0.05). There were no significance

differences between the Oral teaching group and

SSLS group in both the first trial and fifth trial.

Figure 9: Effectiveness of practice sawing in proper slant.

START

END

Check blade angle

Count > pre-set

value

(

)

Show result

Blur level

> pre-set range

Beep & show warning

Error count++

Check blade blur

Moving count ++

YES

NO NO

START

END

START

Select a recorded ex

ert model

Practice

Compare with the model

Show result

Pre-set expert model

Demonstration

Select a record mode

Data record as

an ex

ert model

Flow of practice

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

686

Figure 10: Effectiveness of practice in blur of planning.

6 DISCUSSION

According to these two tests, we found the students

use of the smart phone while sawing and planning to

be very effective in improving their skill. Though

under the condition that subjects were beginners and

the number of subjects was on a small-scale, it

means that beginners can improve their skill of basic

sawing and basic planning without the aid of a

teacher. Even with such tentative results, we believe

this to be meaningful even for distance learning and

enhancing face to face classroom learning. Until

now, e-learning has been used to acquire knowledge

via text and visual aid e.g. videos and YouTube. As

far as using a traditional method, students cannot

understand and assess their skill without a teacher,

i.e. they cannot avoid incorrect posture or improper

movements without a teacher. Because of this

problem, it is difficult for distance learning of skill

based training. However using our method, students

can understand how they should move their body or

tools by themselves. Apparently, we have to validate

our method using more subjects and improving the

precision of data from the sensors. However, this

method of using sensors on a smart phone and the

resulting immediate feedback to students can

enhance traditional learning not only in vocational

education but also physical education and sports.

Even though in this research, the number of subjects

was low and limited, we could get appropriate

results. Our research team has already applied the

approach to about 80 actual junior high school

students. We are going to report the effectiveness of

the practice in detail in a next paper.

7 CONCLUSIONS

In this paper, we developed Self Skill Learning

System (SSLM) using sensors in a smart phone. The

system works on smart phones using Android OS. In

terms of a student’s actual practice, it can indicate

results visually and advise on how to improve. We

conducted sawing and planning experiments that are

basic skills in vocational education. As our results

showed, a group using SSLS could improve both

skills. However as far as this research, we could not

clearly conclude that SSLS was better than a

teacher’s oral method. Currently, we are developing

Skill Learning Management System (SLMS) that

can record students’ results of practice and can

compare those results to other students. Using

SLMS, a teacher will be able to know who is good at

a skill or not before lectures and it will help a

teacher to make a priority of whom he/she teaches to.

ACKNOWLEDGEMENTS

This work was supported by KAKENHI (24730721,

2450169, and 22531009).

REFERENCES

Ando, A., & Abiko, H. (2003). Research of Thing-Making

learning in an elementary school. Bulletins of Miyagi

University of Education, 37, 223-232.

Ando, A., Takaku, T., Aoyagi, A., & Darold, D. (2011).

Development of a teaching material linked with

Twitter for self-instruction to learn skills, Symposium

Mobile 11, 93-96.

Berrett, Dan. (2012). How 'Flipping' the Classroom Can

Improve the Traditional Lecture. Chronicle of Higher

Education. Retrieved from

https://tle.wisc.edu/tleblogs/jhenriqu/how-flipping-

classroom-can-improve-traditional-lecture.

Flip teaching. (n.d.). Retrieved Octorber 21, 2012 from the

Wikipedia: http://en.wikipedia.org/wiki/Flip_teaching

Hattanda, R., Yanai, H., & Kobayashi, K. (2008).

Changing the modes of a mobile phone by adapting to

user actions, Proceedings of Symposium on Mobile

Interactions, 43-46.

Kashiwagi, N., Son, I., & Son, H. (2010). Application of

wireless sensor devices for monitoring learner's

activities, Journal of Japanese Society for Information

and Systems in Education, 24(5), 36-39.

Kato, K., Nagano, K., et al. (Eds.) (2010). Textbook for

Technology Education. Tokyo Shoseki, 59.

Mada, Y., Shioiri M., et al. (Eds.). (2010). Textbook for

Technology Education. Kairyu-do, 57.

The Ministry of Education, Culture, Sports, Science and

Technology (MEXT). (2011). the Guidelines for the

Course of Study, 14.

Yamamoto,T. Yoshino, R., et al. (2007). Consideration of

saw guidance for utilized taping, Bulletins of Saitama

University, 56(2), 15-20.

DevelopmentofaSkillLearningSystemusingSensorsinaSmartPhoneforVocationalEducation

687