EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR
ON COOPERATIVE LEARNING IN AN ENGINEERING COURSE
Krassadaki Evangelia, Matsatsinis F. Nikolaos
Decision Support Systems Laboratory, Department of Production Engineering and Management, Technical University of
Crete, University Campus, 73100, Chania, Greece
Zampetakis A. Leonidas
Management Systems Laboratory, Department of Production Engineering and Management, Technical University of Crete
University Campus, 73100, Chania, Greece
Keywords: Engineering, Higher Education, e-Learning, Cooperative Learning, Transferable Skills.
Abstract: Despite the introduction of new technologies in university education, teaching practice is usually based on
the stereotype of the classroom and the professor who is teaching the students; as a result, e-learning tools
have not been fully integrated. This paper outlines an effort which has started at experimental level by an
engineering department, with the aim to optimally use the digital material of an e-learning platform of
student self-assessment. In this context, we perform various learning activities, we integrate the material
into the activities during class and we use cooperative learning techniques aiming to make an educational
intervention to the learning and social outcomes of education. For this reason we set up an „experiment‟,
which has given the first results, with the participation of undergraduate students who attended a pilot
seminar. Diversification from traditional teaching practices/methods, and the variety of activities in the class
seemed to enhance the performance of students who attended the seminar compared to those who did not
participate. In addition, participants seem to prefer, with regard to the development of their communication
and teamworking skills, the technique of 10-minute group exercises in the class without the use of computer
yet based on the self-assessment material.
1 INTRODUCTION
Although some academics appreciate team and
cooperative work during classes, it is rather an
exception to normalclasses. The usual teaching
practice is indirectly determined by the design of the
classroom, the layout of benches, the organizational
structures, the curricula, the teaching methods, the
general academic culture (Matsaggouras, 1998) and
a series of social characteristics and standards. The
predominant practice is the lecture given by the
professor to the audience composed of a number of
students (Neumann, 2001). An exception to this
educational model is the integrated approach
followed mainly by the universities of Great Britain,
USA and Australia, which set standards both for the
curricula and the skills development of their
students. It is worth mentioning the Problem Based
Learning-PBL educational model, which is applied
in Aalborg University that focuses on the outcomes
and the student. In fact, new technologies and their
applications have enhanced teaching although even
in the most developed countries they have not
managed to change the prevalent feature of
academic classroom and teaching methods (Raptis
and Rapti, 2006).
The Technical University of Crete runs an e-
learning platform (E-Class), since 2005, which is
applied to almost all higher Greek education
institutions. Almost all tutors upload to E-Class
courseware, course notes, exercises and any useful
information regarding their course. Despite the
encouraging comments and the high degree of
satisfaction expressed by students, another research
on the development of the platform shows that the
system is not fully utilized and is mainly used to
download the notes/lectures of courses. This was the
conclusion following observation by the researchers
380
Evangelia K., F. Nikolaos M. and A. Leonidas Z. (2010).
EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR ON COOPERATIVE LEARNING IN AN ENGINEERING COURSE.
In Proceedings of the 2nd International Conference on Computer Supported Education, pages 380-384
Copyright
c
SciTePress
of this paper, in relation to the self-assessment
digital material prepared for the Decision Support
Systems-DSS course, which is a mandatory course
for third-year students at the Department of
Production Engineering and Management.
The E-Class has a special service (exercises) for
academic staff, which is optionally activated so that
the tutor can prepare exercises in the form of
questions-answers for the students. With regard to
the DSS course, the digital material in the form of
self-assessment exercises was divided into groups of
thematic modules, according to the detailed
curriculum and in the form of multiple-choice or
matching or fill-in-blank questions and answers.
This effort aims to enhance teaching of the course,
familiarize students with the further features of the
platform, improve studentsknowledge through self-
assessment and support digital interaction for
educational purposes. However, despite the effort
made at the beginning of each semester by the tutors
to instruct all students attending the course in using
the features of E-Class (2-hour presentation and
relevant notes) and despite the effort to develop,
improve, and update the self-assessment material on
an annual basis it becomes evident that students used
it only to download the material of the course and
hardly ever used the self-assessment feature. The
support team of the course noticed that the self-
assessment material is scarcely used, mainly near the
date of examinations, as a source of possible
questions for the examinations.
In the light of these findings we organized a pilot
seminar, in the form of „experiment‟ involving a
group of twenty-seven volunteer undergraduate
students. The main goal was to utilize the digital
material. Therefore, we prepared a seminar based on
cooperative learning that would use different
techniques for the teaching of the same course. This
seminar was held on different days and hours than
the course, and was attended by 27 volunteers. The
idea of the seminar was to integrate the digital
material in the form of 10-minute group exercises
with the active participation of students. It intended
to measure the final performance of participants
compared to non-participants and to evaluate their
preference in five different activities, one of which
was based on the digital material from E-class. In
particular, the five activities combined with effective
speaking/writing (communication skills) and
teamworking skill, were the following: individual
written assignments (at home), oral presentation
with PowerPoint (prepared at home), group research
projects by 2-3 persons (prepared at home during the
semester; a very well-known alternative given as a
project in the context of the course, and followed by
non-participants in the seminar), active participation
in a discussion in class (questions-answers) and E-
Class exercises (10-minute written exercises in
groups of two in class).
The learning objective of the seminar was not
different from the objective specified in the detailed
curriculum of the course. It is noted that at
undergraduate level, effective writing/speaking and
teamworking are some of the transferable skills
suggested by a large number of researchers (Kemp
and Seagraves, 1995; Venetsanopoulos, 2004;
Baldwin, Cahn, Forman, Lehmann and Wischmeyer,
1979; Caroll, Markauskaite and Calvo, 2008). Both
teams of students (seminar/project) had the
advantage of a final oral examination. In particular,
the second team of students consisted of those who
had chosen to prepare a written research project in
groups of 2 or 3 persons instead of participating in
the seminar. Both teams of students who opted to
participate either in the seminar or the project would
be given an overall performance grade based on:
their performance in the laboratory, their
performance at the seminar (27 persons) or their
grade for the project (57 persons) and their
performance during the oral examination on the
course curriculum.
2 SEMINAR DETAILS
The seminar was held in a classroom equipped with
a round table, in order to promote equality and
familiarity and to avoid the sense of „superiority‟ of
the tutor, as is the case when teaching from the desk,
aiming to encourage participation. The seminar
extended over seven sessions of two hours each,
covering Information Technology Systems and
Decisions Theory, within the educational goals of
the course. The tutor assumed also the role of
facilitator of knowledge not authenticity, coordinator
not transmitter of knowledge, who learns through
the students‟ experiences (Felder, 1996). The role of
students changed from a passive receiver to an equal
participant in the building of knowledge, a producer
of knowledge who draws from previous experiences;
they now assumed an active and interactive, not
passive role. In brief, an effort was made to perform
a more student-centered learning process aiming at
the development of students both at knowledge and
at skills level, and less focused on the usual one-
sided knowledge-based approach which is focused
on the principle of transferring knowledge from the
tutor to the student. In addition, we took into account
EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR ON COOPERATIVE LEARNING IN AN
ENGINEERING COURSE
381
that, as set forth by Yorke and Knight (2004), the
learning process is not merely the proof of
knowledge proficiency in specific cognitive objects
yet a more complex process which informally and
tacitly enhances and improves the experiences and
competences of students. For example, it develops,
strengthens and improves experiences and
competences, such as: communication,
teamworking, etc. For all these reasons, although the
development of students is traditionally linked to
their academic progress, the seminar aimed at a
more global development.
The techniques used by the coordinator of the
seminar included the following:
Brief introduction, usually supported by few slides.
Goal: to avoid monologue.
Brainstorming method through words / sentences /
questions written on the board or distributed to the
participants in printed form. Goal: to connect the
previous experiences-knowledge of students to the
educational objective and obtain their active
participation in the class.
Questions-answers method with relevant material
distributed or oral questions made by the
coordinator. Goal: to obtain the active participation
of students, explore pre-existing knowledge, verify
the usefulness of the course, etc.
Students divided in groups of two, by draw. Goal: to
enhance the cooperation of students, working in
groups of two, who do not necessarily know each
other or are friends; indirectly support team work
with persons not picked by the student
himself/herself.
Short (10-minute) group-written exercises in the
class, derived from the self-assessment material of
E-Class. Goal: to enable the cooperation of the
groups of two students in order to achieve the
objective of the exercise and better learn the material
taught.
Presentation of short group-written exercises. The
exercises were prepared in the class by the groups of
two students and then presented to the audience.
Goal: to facilitate the oral expression of knowledge
obtained by the students and improve their oral
skills.
Individual short written assignments to be prepared
at home referring to the topics covered during the
seminar. The assignments were usually delivered at
the next session. Goal: to improve the knowledge of
students, enable deeper knowledge and further
practice. The coordinator evaluated each individual
written assignment in terms of fully elaborated
subject, presentation of the paper, syntax, spelling,
punctuation, and returned it to the student for
feedback and reflection.
Students selected by draw to make an individual
oral presentation of a topic distributed to them.
Usually, presentations were made every 2 weeks.
Goal: to elaborate the topic, prepare the oral
presentation, structure the presentation in a cohesive
and comprehensive manner within the set deadline.
3 RESULTS
In analyzing the results of the pilot initiative we
checked the performance of students and we
delivered a specially formulated questionnaire at the
end of the semester to the participants in the
seminar.
In Fig. 1 we present the performance of both
teams of students on a scale from 0 to 10 (excellent)
[pass grades are from 5 to 10]. We used independent
samples, t-tests, to determine if there were
significant differences in student performance. Only
the performance in the final oral examination was
found to be statistically significant [t (66.33) = 2.12,
p <0.05)]. The effect size, Cohen‟s d, was found to
be 0.46, which is a moderate effect. Furthermore, we
found no statistical differences in performance based
on the students‟ gender.
With regard to the teaching techniques and
methods, we examined their effect in enhancing
effective writing, effective speaking and
teamworking. This research was carried out with a
specially formulated questionnaire using the
Conjoint Analysis (CA) technique. CA (also known
as “trade off analysis”) is a multivariate technique
used to estimate or determine how respondents
develop preferences for products/services. It is
widely used in marketing research and is based on
the premise that consumers assess the value of a
product by combining the separate amounts of value
provided by each attribute (or factor) of the
particular product or service.
In the CA experiment conducted in this study,
the “product” is teaching techniques, which is
decomposed into relevant factors that can be
combined to fully describe them. From the specified
factors and factor levels, hypothetical products
(scenarios) were constructed for students to evaluate.
The 5 specific factors and factor levels (in
parentheses) were as follows: 1. preparation of
individual written assignments at home (every week
or every two weeks), 2. individual presentation of
the assignment using PowerPoint in the class (every
week or every two weeks), 3. preparation of a group
CSEDU 2010 - 2nd International Conference on Computer Supported Education
382
research project by 2-3 persons during the semester
(yes/no), 4. active participation through questions-
answers during the course (yes/no), 5. group written
exercises in class based on digital material (yes/no).
The combination of the five factors and two
levels per factor gave rise to 32 possible scenarios (2
x 2 x 2 x 2 x 2). To reduce the number of profiles to
a manageable size, while at the same time
maintaining randomness, a fractional factorial
design using SPSS was used. This process reduced
the number of profiles to be evaluated to ten (see
Table 1), including two holdout profiles. Holdouts
are scenarios which are rated by the students but are
not used to build the preference model. Instead, they
are used as a check on the internal validity of the
model. The full-profile approach, whereby students
were asked to rate the full descriptions according to
preference, was used. A rating scale anchored from
one to seven, with 1 being “no improvement” and 7
“great improvement” was selected. The study
utilized a verbal description approach. The ten
scenarios used in the questionnaire had the following
form: (scenario 1) Suppose that every week you
must deliver an individual short written assignment
and every 2 weeks you must prepare an oral
presentation on PowerPoint for the purposes of the
course. You will not be given any group research
projects. In class you will actively participate by
answering questions made by the coordinator for
discussion, and you will prepare together with your
classmates short written exercises that facilitate the
learning process. Do you think that the above
activities improve your written and oral expression
and your ability to work in a team? The SPSS
conjoint procedure (SPSS, 2004) was used to
calculate utility scores (or part-worths) for each
individual respondent and for the whole sample.
These utility scores are analogous to regression
coefficients, and are used to estimate the relative
importance of each factor. According to the results
of preference expressed by students, the five factors
rank as follows in terms of importance: 1. short
group exercises in class (E-Class material)
(29.37%); 2. active participation through questions-
answers during the course (26.12%); 3. group
research projects during the semester (21.89%); 4.
preparation of individual written assignments at
home (14.26%), and 5. individual presentation of the
assignment on PowerPoint in class (8.37%).
Our results indicate an adequate level of internal
validity in terms of Pearson‟s product moment
correlation coefficient (r = 0.98, p<0.001), and
Kendall‟s tau = 0.997, p< 0.001) for the whole
sample. With regard to the means used to improve
written-oral expression and cooperation with other
students, there seems to be a preference firstly to
exercises performed in the classroom, based on E-
Class self-assessment digital material, then to
discussion through questions made by the
coordinator, etc.
4 CONCLUSIONS
Although the sample was very small, the initial
research indicated whether tutors can affect the
results of learning by applying a mix of teaching
methods and techniques. The performance in the
final oral examination of the students who
participated in the seminar was higher compared to
the performance of non-participants. It is noted that
the written comments of participants and the high
level of satisfaction expressed by the students
encouraged us to repeat the seminar the following
year and thus achieve higher reliability of our
conclusions. It is worth mentioning that although
students did not generally use the self-assessment
digital material, in the form of 10-minute long
exercises, it was highlighted as the most important
tool to improve the two transferable skills,
communication and teamworking. McDonald also
reached a similar conclusion in 1977 when he
noticed that a significant part of learning was taking
place in group discussions, away from computers yet
regarding computer work.
We do not know whether students would keep the
same order of preference if written group exercises
in class were diversified in connection with the
digital material and had a different content or form.
Prima facie we can conclude that students developed
a better understanding of the usefulness and value of
the self-assessment digital material, for the
additional reason that it promoted the interaction,
exchange and negotiation of ideas both between the
groups of two students and at group level in the
class. Additional conclusions came to light: it
became evident that the theoretical part of an
academic course can become more „exciting‟ and
interesting to students if a mix of teaching
techniques is used. In this context, numerous tools,
techniques and methods can be applied, and any
digital means can, directly or indirectly, become part
of everyday practice and be useful.
EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR ON COOPERATIVE LEARNING IN AN
ENGINEERING COURSE
383
Figure 1: Comparing the performance of student groups.
Table 1: Scenarios through Fractional Factorial Analysis.
s/n
Individual written
assignments
Oral presentation using
PowerPoint
Group research project
during the semester
Discussion through
questions-answers
Short exercises in
groups of two
Status
1
Every week
Every 2 weeks
No
Yes
Yes
Holdout
2
Every 2 weeks
Every 2 weeks
Yes
Yes
Yes
Design
3
Every week
Every week
No
No
Yes
Design
4
Every 2 weeks
Every 2 weeks
No
No
Yes
Design
5
Every week
Every week
Yes
Yes
Yes
Design
6
Every 2 weeks
Every 2 weeks
Yes
No
No
Holdout
7
Every week
Every 2 weeks
Yes
No
No
Design
8
Every 2 weeks
Every week
No
Yes
No
Design
9
Every week
Every 2 weeks
No
Yes
No
Design
10
Every 2 weeks
Every week
Yes
No
No
Design
REFERENCES
Matsagouras, Η., 1998. Strategies of teaching. The critical
thought in the teaching practice. Ed. Gutenberg (in
Greek), 2
nd
edition, Athens.
Raptis, Α., Α. Rapti, 2006. Learning and teaching during
the information technology society. Vol. Α‟. Ed.
Raptis (in Greek), Athens.
Baldwin, J.C., C.R. Cahn, J.W. Forman, H. Lehmann,
C.R. Wischmeyer, 1979. A Model Undergraduate
Electrical Engineering Curriculum. IEEE Transaction
on Education, vol. E-22(2).
Caroll, L.N., L. Markauskaite and R. A. Calvo, 2008. E-
Portfolios for developing tranferable Skills in a
Freshman Engineering Course. IEEE Transaction on
Education, 50(4).
Felder, R.M., 1996. Matters of style. ASEE Prism, 6(4),
18-32.
Kemp, J. and L. Seagraves, 1995. Transferable Skills-can
higher education deliver? Studies in Higher Education,
20(3), 315-328.
McDonald, B., 1977. The educational evaluation of
NDPCAL, British Journal of Educational Technology,
8(3).
Neumann, R., 2001. Disciplinary differences and
university teaching. Studies in Higher Education,
26(2), 135-146.
Venetsanopoulos, A., 2004. Advice to Young Engineers,
IEEE Signal Processing Magazine, vol. July.
7
7
7
0
1
2
3
4
5
6
7
8
9
10
Average laboratory work grade
Average seminar/project
grade
Average Oral Grade
Average Overall Performance
Students with project
Seminar students
CSEDU 2010 - 2nd International Conference on Computer Supported Education
384