EXPERIMENT WITH PERSONAL RESPONSE DEVICES

Advantages and Drawbacks Identified in Engineering Studies

F. R. Lara

, E. Herruzo

, F. J. Jiménez

, F. M. Pérez

and C. Corredor

Higher Polytechnic School of Córdoba, Córdoba University, Campus de Rabanales, Córdoba, Spain

Higher Technical School of Industrial Engineering, Málaga University, Málaga, Spain

Keywords: Personal Response Devices, Engineering, Experiment, Advantages, Drawbacks.

Abstract: This work presents the main results and conclusions drawn from a teaching experiment, in the studies of

Mechanical Engineering from the Polytechnic School of Córdoba University (Spain), in the 2010 to 2011

course. This experiment was to use extensively, in one of the theory groups, an interactive system based on

personal response devices. The main purpose of the meeting was to compare the results with other group in

which the system was not used. Also we wanted to know the opinion of the teachers and students

participating in this experiment. In this way we could know both the advantages and the disadvantages of its

implementation, the technical dificulties and if it is a really useful teaching tool. The results can be

interesting for teachers and educational supervisors who want to know the advantages and limitations of

these systems.

1 INTRODUCTION

The implementation of the European Higher

Education Area is a new approach for teaching and

assessment methodology, in which the teacher must

focus their efforts on value, not only the students'

level of knowledge, but also the so-called

competences (Goñi, 2005). The acquisition of

contents and competences by the students is largely

based on activities and teaching methods applied in

the classroom throughout the course (Herruzo,

2005). Any action designed to facilitate both the

development and evaluation of these tasks will be

well received. On the other hand, any face activity

requires an active and continued involvement of

students. Students can remain neither passive nor

non-development of exhibitions and activities

proposed. Any resource that encourages the students'

participation will be equally well considered.

There are several works related to the personal

response devices, also called clickers (Morling 2008,

Shaffer 2009). All of them develop experiments

about different teaching methodology and indicate

the benefits from this technology. This paper is

organized as follows. Section 2 indicates the

objectives of the experiment. Section 3 presents the

main features of the system. In section 4 the

experiment is described. Section 5 shows the

assessment of the experiment.

Finally we draw some

conclusions.

2 OBJECTIVES OF THE

EXPERIMENT

The development of this experiment was intended to

cover the following main objectives:

Promoting participation. The current reality of

university students, at least in the target group of this

pilot scheme, is that they are mere spectators of the

presentations made by the teacher (Ellis, 2005). The

little or no student participation is precisely one of

the main complaints of the teachers involved.

Therefore, one of the main objectives of this

experiment was trying to encourage such

participation in classroom activities scheduled.

Continuous assessment facilitators. The teacher need

to have the mechanisms that would allow a

continuous assessment throughout the course. These

tools are not easy to find in traditional

methodologies and resources (Waters, 1997).

Improving academic performance. Another objective

was to check, in situ, the students content

assimilation level in each face to face session. This

suggests that the correct implementation could

trigger a reasonable improvement of students'

166

R. Lara F., Herruzo E., J. Jiménez F., M. Pérez F. and Corredor C..

EXPERIMENT WITH PERSONAL RESPONSE DEVICES - Advantages and Drawbacks Identiﬁed in Engineering Studies.

DOI: 10.5220/0003919701660169

In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 166-169

ISBN: 978-989-8565-07-5

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

academic performance object of the experiment.

Comparison of the results. Another purpose of

this work was an analysis and comparison of the

academic results with the group that did not use the

system. It was also to detect their main advantages

and drawbacks and its outstanding implementation

difficulties. This would give an overview about the

advisability of its use in courses and related degree

programs.

3 MAIN FEATURES OF THE

SYSTEM

The teacher asks some questions during the session

and the students answer them together using a

wireless remote control (clicker) that has been

individually assigned. The global answers can be

visualized instantly during the session; the

individual assignment of each clicker allows a later

assessment of the students.

3.1 Advantages

The main advantages of this implantation are:

The increase of attention. According with different

researches, the level of student attention shows ups

and downs during a typical master class.

The increase in the level of participation. An active

participation of the students is needed during the

face to face session where different questions are

asked and consistent answers are demanded. It is

possible to build up some questions with fixed close

answers, test the general opinion or open a debate.

The improvement in the withholding of the

concepts. It is boosted the withholding of the

concepts giving immediately the right answer for

each question asked. It also allows the teachers to

know the knowledge level about the subject and to

decide about if the less successful contents ought to

be reinforced or repeated.

Continued assessment. The continued use of the

system and the later individually results treatment

allows easily to administer the knowledge level and

to get competences in a continuous assessment

system.

3.2 Difficulties

The main difficulties are:

Considerable initial investment. To carry out an

experiment that it is opened to a Degree or a Study

Centre, the inversion cost can be higher due to the

number of clickers that are needed.

Users need training. The methodology of teaching

that is used, to obtain their maximum result as a

pedagogical tool, can be complicated. This tool has

to be adapted to the every teaching by teachers in

their lesson (Barker, 1993).

Logistic control and adaptation to spaces. Classroom

with enough equipment and computer supports

should be available. A permanent staff for the

maintenance and coordinating of the system has to

be designated. Its implementation can be a full

failure, and all its advantages can be lost, if these

risks are not taken into account.

4 DESCRIPTION OF THE

EXPERIMENT

As it has been said, a very important aspect that it

has to be taken into account for a successful result it

is the appropriate training for the teachers involved.

In this case, the training had been carried out in

several sessions. A first session to inform about the

experiment and the resources that are needed to

develop it was carried out. Then, others lessons with

technical character were developed. In these classes

the participants were instructed about the use of the

tool, the main advantage, its applications and the

possible use during the class.

To obtain the equipment and software for the

first implementation, several innovation projects

were carried out, both the Polytechnic School of

Córdoba University (Spain) and the Vice President

for Quality and Innovation at the University of

Córdoba provided the financial recourses.

The implementation was carried out, at the

beginning, in the first year of the Engineering

Mechanical Degree from the Polytechnic School of

Córdoba University (Spain) and only in one of the

two theoretical lesson groups. In this way, the results

obtained could be compared with the blank group,

about the same lessons and the same teachers.

To coordinate the results and to make easy the

use of the system, an individual assignment was

carried out, so each student could use the same

remote controller in every lesson, which it allowed

to know globally the evolution of learning.

At the end of the year, a survey to the teacher

who participated in the experiment was carried out,

through an online test that it included questions as

general character about the participation,

performance or tool’s use, as other with a more

specific character as the suitability in his interests,

kind of use that was did.

In a same way students were pulled. In this case the

EXPERIMENTWITHPERSONALRESPONSEDEVICES-AdvantagesandDrawbacksIdentifiedinEngineering

Studies

167

test was carried out in a face-to-face way, the

interactive controller, what it served to analyze the

some of the result there, helped us, it allowed the

participation and comments of the audience.

5 ASSESSMENT

In this section, some of the main questions asked in

the questionnaires will be presented. Not only those

carried out by the teachers, but also the ones directed

to the students.

A further analysis will be devoted to the obtained

results in the academic field as well as in the other

aspects assessed. These results will be compared

with those of the other group and the final

conclusions will be exposed.

5.1 Questionnaire for Teachers

The questions asked to the teachers participating in

the experiment were divided into several blocks,

related to methodological aspects, the evaluation

criteria applied as well as others which trying to

know if the interactive control system served as a

tool for evaluation and qualification. The aim was to

deduce whether the results were comparable in both

groups. Another blocks of questions were focused to

find out the goodness or adequacy of the system to

the type of subject being dealt with; the difficulties

come across for its application in the classroom and

the teachers and students general impressions on

these tools, as well as some questions related to the

satisfaction on the management and coordination

during the use of the system.

5.2 Questionnaire for Students

The questions were divided, as in the previous case,

in several blocks. The first one the students’

impressions about how they were personally

affected by the use of the tool from the improving

learning viewpoint were analyzed. A second block

was focused on the use of the system as evaluation

tool and the perception that students had on the

usefulness and effectiveness in this evaluation.

Finally, the students’ opinion was sought in

relation to the use of the tool that the participant

teachers had created relation to aspects of

coordination, management skills, as well as using it

to improve teaching quality in general.

5.3 Analysis of Results

The results obtained in the questionnaires already

mentioned and the views expressed by the

participating members enable us to clearly glimpse

into the benefits and limitations of using such tools.

They are not as conclusive, however, in the

aspect of academic performance, mainly due to the

already mentioned constraints, and because only

25% of teachers applied the same criteria for

evaluation and teaching methods in both target

groups.

In this regard and with the reservations already

mentioned, it must be said that only in the subjects

in which the system was used as a tool of teaching

support, reinforcement of principal ideas, enabling

and encouraging student participation, academic

performance was significantly higher.

In the cases where it was used as an evaluation

tool, the results were very similar or even worse in

the target group compared to the blank group.

5.3.1 Analysis of Teacher Survey

The 70% of the teachers used the tool as a means of

testing the level of content knowledge and

understanding of students or to support and reinforce

key ideas, but they did not considered it as an

evaluation and rating tool. Practically the total

number out of this 70% agreed that students had a

good impression on the use of the tool. Moreover,

they believed that its use did help the students get a

better grade.

A 30% of users stated that students did not

perceive it in a positive way and they did not think it

helped the students get a better grade in their

subject.

A 25% believed that it was difficult to adapt the

use of interactive controls to their commonly used

teaching methodology, another 15% did not find it

very difficult and the remaining 60% considered it

easy or very easy. All in all, the 100% were

interested in using it again in future.

5.3.2 Analysis of Student Survey

Nearly 100% of the students appreciated the use of

new technologies by the teachers. However,

approximately 40% had poor or very poor

impression on the use of clickers, 50% did not have

a very clear idea on the issue and only 10%

considered it good or very good. Their further

arguments were that they were reluctant to be

constantly evaluated or graded, as they seemed to

understand when using the clickers regularly.

On the other hand, over 75% of them believed

that the use of the controls forced them to be more

attentive and participative in class and also it

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

168

allowed them to refresh and set key ideas.

However 60% did not think it favored the

process of learning, which is contradictory with the

previous statement. Once discussed these result with

the students, they believed that teachers in general

had failed to properly integrate the tool in the

teaching methodology of the subjects. In fact, one

question focused specifically on whether teachers

need better preparation as well as coordination to

unify criteria and the vast majority, over 85%,

thought so.

Nearly 80% stated that they would have got

better grades if they had not used the system of

controls and that they preferred the use of controls

not as an evaluation tool, but as tool to support

teaching.

To sum up:

 The students’ perception is that the teachers

have failed to properly integrate the tool and

adapt it successfully to their methodology.

 While recognizing that the use of controls has

forced them to be more attentive and

participatory and even has helped them retain

the key concepts, they do not like the idea of

being constantly evaluated.

 They reject the use of controls as an

assessment tool, substituting the traditional

methods. In fact, they are convinced that they

would have obtained better grades without the

use of the system.

 Finally, they consider their teachers need more

preparation and coordination for proper use of

the system.

6 CONCLUSIONS

It is not easy to establish definitive conclusions by

comparing the results between the two groups

studied, especially regarding academic issues.

The main conclusions are:

 This type of interactive tools, like any others,

require proper training to be used by teachers,

not just technical but also methodological and

adjustment to each subject (Salinas, 2004).

 Its usefulness is demonstrated as a catalyst,

encouraging the participation of students and

their activation in the face to face sessions.

 It is a tool that improves content retention as

well as key ideas.

 Clearly, if the equipment is used only as an

evaluation tool, the perception of students is

not satisfactory, creating a rejection that

cripples its methodological qualities.

Therefore, it appears not to be used as a

substitute for conventional evaluation tools,

but as a complement to them.

 Despite the difficulties in the implementation

of any system for the first time, virtually all of

the teachers would like to reuse it in future

times, which shows its good sense in this

group.

 The students, meanwhile, only show a high

degree of satisfaction and they want to use it

when the system is used purely for educational

purposes and not as an evaluation tool.

 This tool has a great potential in the

improving of methodological and didactic

aspects. It is easily adaptable to most subjects

and contents and it can be integrated and

extended without too much effort to a whole

Course, Degree Program or Study Centre.

REFERENCES

Goñi, J., Zabala, J., 2005. El espacio europeo de

educación superior, un reto para la Universidad:

competencias, tareas y evaluación. Octaedro,

Barcelona.

Herruzo, E., Climent, M. S., y otros, 2005.

Implementación experimental del sistema ECTS en la

titulaciones de Ing. Téc. Ind. En la especialidad de

Electricidad, Electrónica y Mecánica. Servicio de

Publicaciones Universidad de Córdoba. Córdoba.

Ellis, G. W., Rudnitsky, A. N., Scordilis, G. E, 2005.

Finding meaning in the classroom: Learner-centered

approaches that engage students in engineering.

International Journal of Engineering Education.

21(6): 1148-1158.

Waters, R., McCracken, M., 1997. Assessment and

Evaluation in Problem-Based Learning. The 27

Frontiers in Education Conference.

Barker, B. O., 1993. Using Instructional Technologies in

the Preparation of Teachers for the 21

Century.

Conferencia presentada en la “National Conference

on Creating the Quality School”, ERIC Document

Reproduction Service, Oklahoma City.

Salinas, J., Aguaded, J., Cabero, J., 2004. Tecnologías

para la educación. Diseño, producción y evaluación

de medios para la formación docente. Alianza

Editorial. Madrid.

Morling, B., McAuliffe, M., et al. 2008. Efficacy of

Personal Response Systems (“Clickers”) in Large

Introductory Psychology Classes. Teaching of

Phychology. Vol. 35, pp. 45-50. ISSN: 0098-6283.

Shaffer, D. M., Collura, M. J., 2009. Evaluating the

effectiveness of a personal response system in the

classroom. Teaching of Phychology. Vol. 36, no. 4, pp.

273-277. ISSN: 0098-6283.

EXPERIMENTWITHPERSONALRESPONSEDEVICES-AdvantagesandDrawbacksIdentifiedinEngineering

Studies

169