EXPERIMENT WITH PERSONAL RESPONSE DEVICES
Advantages and Drawbacks Identified in Engineering Studies
F. R. Lara
1
, E. Herruzo
1
, F. J. Jiménez
1
, F. M. Pérez
2
and C. Corredor
1
1
Higher Polytechnic School of Córdoba, Córdoba University, Campus de Rabanales, Córdoba, Spain
2
Higher Technical School of Industrial Engineering, Málaga University, Málaga, Spain
Keywords: Personal Response Devices, Engineering, Experiment, Advantages, Drawbacks.
Abstract: This work presents the main results and conclusions drawn from a teaching experiment, in the studies of
Mechanical Engineering from the Polytechnic School of Córdoba University (Spain), in the 2010 to 2011
course. This experiment was to use extensively, in one of the theory groups, an interactive system based on
personal response devices. The main purpose of the meeting was to compare the results with other group in
which the system was not used. Also we wanted to know the opinion of the teachers and students
participating in this experiment. In this way we could know both the advantages and the disadvantages of its
implementation, the technical dificulties and if it is a really useful teaching tool. The results can be
interesting for teachers and educational supervisors who want to know the advantages and limitations of
these systems.
1 INTRODUCTION
The implementation of the European Higher
Education Area is a new approach for teaching and
assessment methodology, in which the teacher must
focus their efforts on value, not only the students'
level of knowledge, but also the so-called
competences (Goñi, 2005). The acquisition of
contents and competences by the students is largely
based on activities and teaching methods applied in
the classroom throughout the course (Herruzo,
2005). Any action designed to facilitate both the
development and evaluation of these tasks will be
well received. On the other hand, any face activity
requires an active and continued involvement of
students. Students can remain neither passive nor
non-development of exhibitions and activities
proposed. Any resource that encourages the students'
participation will be equally well considered.
There are several works related to the personal
response devices, also called clickers (Morling 2008,
Shaffer 2009). All of them develop experiments
about different teaching methodology and indicate
the benefits from this technology. This paper is
organized as follows. Section 2 indicates the
objectives of the experiment. Section 3 presents the
main features of the system. In section 4 the
experiment is described. Section 5 shows the
assessment of the experiment.
Finally we draw some
conclusions.
2 OBJECTIVES OF THE
EXPERIMENT
The development of this experiment was intended to
cover the following main objectives:
Promoting participation. The current reality of
university students, at least in the target group of this
pilot scheme, is that they are mere spectators of the
presentations made by the teacher (Ellis, 2005). The
little or no student participation is precisely one of
the main complaints of the teachers involved.
Therefore, one of the main objectives of this
experiment was trying to encourage such
participation in classroom activities scheduled.
Continuous assessment facilitators. The teacher need
to have the mechanisms that would allow a
continuous assessment throughout the course. These
tools are not easy to find in traditional
methodologies and resources (Waters, 1997).
Improving academic performance. Another objective
was to check, in situ, the students content
assimilation level in each face to face session. This
suggests that the correct implementation could
trigger a reasonable improvement of students'
166
R. Lara F., Herruzo E., J. Jiménez F., M. Pérez F. and Corredor C..
EXPERIMENT WITH PERSONAL RESPONSE DEVICES - Advantages and Drawbacks Identified in Engineering Studies.
DOI: 10.5220/0003919701660169
In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 166-169
ISBN: 978-989-8565-07-5
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
academic performance object of the experiment.
Comparison of the results. Another purpose of
this work was an analysis and comparison of the
academic results with the group that did not use the
system. It was also to detect their main advantages
and drawbacks and its outstanding implementation
difficulties. This would give an overview about the
advisability of its use in courses and related degree
programs.
3 MAIN FEATURES OF THE
SYSTEM
The teacher asks some questions during the session
and the students answer them together using a
wireless remote control (clicker) that has been
individually assigned. The global answers can be
visualized instantly during the session; the
individual assignment of each clicker allows a later
assessment of the students.
3.1 Advantages
The main advantages of this implantation are:
The increase of attention. According with different
researches, the level of student attention shows ups
and downs during a typical master class.
The increase in the level of participation. An active
participation of the students is needed during the
face to face session where different questions are
asked and consistent answers are demanded. It is
possible to build up some questions with fixed close
answers, test the general opinion or open a debate.
The improvement in the withholding of the
concepts. It is boosted the withholding of the
concepts giving immediately the right answer for
each question asked. It also allows the teachers to
know the knowledge level about the subject and to
decide about if the less successful contents ought to
be reinforced or repeated.
Continued assessment. The continued use of the
system and the later individually results treatment
allows easily to administer the knowledge level and
to get competences in a continuous assessment
system.
3.2 Difficulties
The main difficulties are:
Considerable initial investment. To carry out an
experiment that it is opened to a Degree or a Study
Centre, the inversion cost can be higher due to the
number of clickers that are needed.
Users need training. The methodology of teaching
that is used, to obtain their maximum result as a
pedagogical tool, can be complicated. This tool has
to be adapted to the every teaching by teachers in
their lesson (Barker, 1993).
Logistic control and adaptation to spaces. Classroom
with enough equipment and computer supports
should be available. A permanent staff for the
maintenance and coordinating of the system has to
be designated. Its implementation can be a full
failure, and all its advantages can be lost, if these
risks are not taken into account.
4 DESCRIPTION OF THE
EXPERIMENT
As it has been said, a very important aspect that it
has to be taken into account for a successful result it
is the appropriate training for the teachers involved.
In this case, the training had been carried out in
several sessions. A first session to inform about the
experiment and the resources that are needed to
develop it was carried out. Then, others lessons with
technical character were developed. In these classes
the participants were instructed about the use of the
tool, the main advantage, its applications and the
possible use during the class.
To obtain the equipment and software for the
first implementation, several innovation projects
were carried out, both the Polytechnic School of
Córdoba University (Spain) and the Vice President
for Quality and Innovation at the University of
Córdoba provided the financial recourses.
The implementation was carried out, at the
beginning, in the first year of the Engineering
Mechanical Degree from the Polytechnic School of
Córdoba University (Spain) and only in one of the
two theoretical lesson groups. In this way, the results
obtained could be compared with the blank group,
about the same lessons and the same teachers.
To coordinate the results and to make easy the
use of the system, an individual assignment was
carried out, so each student could use the same
remote controller in every lesson, which it allowed
to know globally the evolution of learning.
At the end of the year, a survey to the teacher
who participated in the experiment was carried out,
through an online test that it included questions as
general character about the participation,
performance or tool’s use, as other with a more
specific character as the suitability in his interests,
kind of use that was did.
In a same way students were pulled. In this case the
EXPERIMENTWITHPERSONALRESPONSEDEVICES-AdvantagesandDrawbacksIdentifiedinEngineering
Studies
167
test was carried out in a face-to-face way, the
interactive controller, what it served to analyze the
some of the result there, helped us, it allowed the
participation and comments of the audience.
5 ASSESSMENT
In this section, some of the main questions asked in
the questionnaires will be presented. Not only those
carried out by the teachers, but also the ones directed
to the students.
A further analysis will be devoted to the obtained
results in the academic field as well as in the other
aspects assessed. These results will be compared
with those of the other group and the final
conclusions will be exposed.
5.1 Questionnaire for Teachers
The questions asked to the teachers participating in
the experiment were divided into several blocks,
related to methodological aspects, the evaluation
criteria applied as well as others which trying to
know if the interactive control system served as a
tool for evaluation and qualification. The aim was to
deduce whether the results were comparable in both
groups. Another blocks of questions were focused to
find out the goodness or adequacy of the system to
the type of subject being dealt with; the difficulties
come across for its application in the classroom and
the teachers and students general impressions on
these tools, as well as some questions related to the
satisfaction on the management and coordination
during the use of the system.
5.2 Questionnaire for Students
The questions were divided, as in the previous case,
in several blocks. The first one the students’
impressions about how they were personally
affected by the use of the tool from the improving
learning viewpoint were analyzed. A second block
was focused on the use of the system as evaluation
tool and the perception that students had on the
usefulness and effectiveness in this evaluation.
Finally, the students’ opinion was sought in
relation to the use of the tool that the participant
teachers had created relation to aspects of
coordination, management skills, as well as using it
to improve teaching quality in general.
5.3 Analysis of Results
The results obtained in the questionnaires already
mentioned and the views expressed by the
participating members enable us to clearly glimpse
into the benefits and limitations of using such tools.
They are not as conclusive, however, in the
aspect of academic performance, mainly due to the
already mentioned constraints, and because only
25% of teachers applied the same criteria for
evaluation and teaching methods in both target
groups.
In this regard and with the reservations already
mentioned, it must be said that only in the subjects
in which the system was used as a tool of teaching
support, reinforcement of principal ideas, enabling
and encouraging student participation, academic
performance was significantly higher.
In the cases where it was used as an evaluation
tool, the results were very similar or even worse in
the target group compared to the blank group.
5.3.1 Analysis of Teacher Survey
The 70% of the teachers used the tool as a means of
testing the level of content knowledge and
understanding of students or to support and reinforce
key ideas, but they did not considered it as an
evaluation and rating tool. Practically the total
number out of this 70% agreed that students had a
good impression on the use of the tool. Moreover,
they believed that its use did help the students get a
better grade.
A 30% of users stated that students did not
perceive it in a positive way and they did not think it
helped the students get a better grade in their
subject.
A 25% believed that it was difficult to adapt the
use of interactive controls to their commonly used
teaching methodology, another 15% did not find it
very difficult and the remaining 60% considered it
easy or very easy. All in all, the 100% were
interested in using it again in future.
5.3.2 Analysis of Student Survey
Nearly 100% of the students appreciated the use of
new technologies by the teachers. However,
approximately 40% had poor or very poor
impression on the use of clickers, 50% did not have
a very clear idea on the issue and only 10%
considered it good or very good. Their further
arguments were that they were reluctant to be
constantly evaluated or graded, as they seemed to
understand when using the clickers regularly.
On the other hand, over 75% of them believed
that the use of the controls forced them to be more
attentive and participative in class and also it
CSEDU2012-4thInternationalConferenceonComputerSupportedEducation
168
allowed them to refresh and set key ideas.
However 60% did not think it favored the
process of learning, which is contradictory with the
previous statement. Once discussed these result with
the students, they believed that teachers in general
had failed to properly integrate the tool in the
teaching methodology of the subjects. In fact, one
question focused specifically on whether teachers
need better preparation as well as coordination to
unify criteria and the vast majority, over 85%,
thought so.
Nearly 80% stated that they would have got
better grades if they had not used the system of
controls and that they preferred the use of controls
not as an evaluation tool, but as tool to support
teaching.
To sum up:
The students’ perception is that the teachers
have failed to properly integrate the tool and
adapt it successfully to their methodology.
While recognizing that the use of controls has
forced them to be more attentive and
participatory and even has helped them retain
the key concepts, they do not like the idea of
being constantly evaluated.
They reject the use of controls as an
assessment tool, substituting the traditional
methods. In fact, they are convinced that they
would have obtained better grades without the
use of the system.
Finally, they consider their teachers need more
preparation and coordination for proper use of
the system.
6 CONCLUSIONS
It is not easy to establish definitive conclusions by
comparing the results between the two groups
studied, especially regarding academic issues.
The main conclusions are:
This type of interactive tools, like any others,
require proper training to be used by teachers,
not just technical but also methodological and
adjustment to each subject (Salinas, 2004).
Its usefulness is demonstrated as a catalyst,
encouraging the participation of students and
their activation in the face to face sessions.
It is a tool that improves content retention as
well as key ideas.
Clearly, if the equipment is used only as an
evaluation tool, the perception of students is
not satisfactory, creating a rejection that
cripples its methodological qualities.
Therefore, it appears not to be used as a
substitute for conventional evaluation tools,
but as a complement to them.
Despite the difficulties in the implementation
of any system for the first time, virtually all of
the teachers would like to reuse it in future
times, which shows its good sense in this
group.
The students, meanwhile, only show a high
degree of satisfaction and they want to use it
when the system is used purely for educational
purposes and not as an evaluation tool.
This tool has a great potential in the
improving of methodological and didactic
aspects. It is easily adaptable to most subjects
and contents and it can be integrated and
extended without too much effort to a whole
Course, Degree Program or Study Centre.
REFERENCES
Goñi, J., Zabala, J., 2005. El espacio europeo de
educación superior, un reto para la Universidad:
competencias, tareas y evaluación. Octaedro,
Barcelona.
Herruzo, E., Climent, M. S., y otros, 2005.
Implementación experimental del sistema ECTS en la
titulaciones de Ing. Téc. Ind. En la especialidad de
Electricidad, Electrónica y Mecánica. Servicio de
Publicaciones Universidad de Córdoba. Córdoba.
Ellis, G. W., Rudnitsky, A. N., Scordilis, G. E, 2005.
Finding meaning in the classroom: Learner-centered
approaches that engage students in engineering.
International Journal of Engineering Education.
21(6): 1148-1158.
Waters, R., McCracken, M., 1997. Assessment and
Evaluation in Problem-Based Learning. The 27
th
Frontiers in Education Conference.
Barker, B. O., 1993. Using Instructional Technologies in
the Preparation of Teachers for the 21
st
Century.
Conferencia presentada en la “National Conference
on Creating the Quality School”, ERIC Document
Reproduction Service, Oklahoma City.
Salinas, J., Aguaded, J., Cabero, J., 2004. Tecnologías
para la educación. Diseño, producción y evaluación
de medios para la formación docente. Alianza
Editorial. Madrid.
Morling, B., McAuliffe, M., et al. 2008. Efficacy of
Personal Response Systems (“Clickers”) in Large
Introductory Psychology Classes. Teaching of
Phychology. Vol. 35, pp. 45-50. ISSN: 0098-6283.
Shaffer, D. M., Collura, M. J., 2009. Evaluating the
effectiveness of a personal response system in the
classroom. Teaching of Phychology. Vol. 36, no. 4, pp.
273-277. ISSN: 0098-6283.
EXPERIMENTWITHPERSONALRESPONSEDEVICES-AdvantagesandDrawbacksIdentifiedinEngineering
Studies
169