EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR

ON COOPERATIVE LEARNING IN AN ENGINEERING COURSE

Krassadaki Evangelia, Matsatsinis F. Nikolaos

Decision Support Systems Laboratory, Department of Production Engineering and Management, Technical University of

Crete, University Campus, 73100, Chania, Greece

Zampetakis A. Leonidas

Management Systems Laboratory, Department of Production Engineering and Management, Technical University of Crete

University Campus, 73100, Chania, Greece

Keywords: Engineering, Higher Education, e-Learning, Cooperative Learning, Transferable Skills.

Abstract: Despite the introduction of new technologies in university education, teaching practice is usually based on

the stereotype of the classroom and the professor who is teaching the students; as a result, e-learning tools

have not been fully integrated. This paper outlines an effort which has started at experimental level by an

engineering department, with the aim to optimally use the digital material of an e-learning platform of

student self-assessment. In this context, we perform various learning activities, we integrate the material

into the activities during class and we use cooperative learning techniques aiming to make an educational

intervention to the learning and social outcomes of education. For this reason we set up an „experiment‟,

which has given the first results, with the participation of undergraduate students who attended a pilot

seminar. Diversification from traditional teaching practices/methods, and the variety of activities in the class

seemed to enhance the performance of students who attended the seminar compared to those who did not

participate. In addition, participants seem to prefer, with regard to the development of their communication

and teamworking skills, the technique of 10-minute group exercises in the class without the use of computer

yet based on the self-assessment material.

1 INTRODUCTION

Although some academics appreciate team and

cooperative work during classes, it is rather an

exception to „normal‟ classes. The usual teaching

practice is indirectly determined by the design of the

classroom, the layout of benches, the organizational

structures, the curricula, the teaching methods, the

general academic culture (Matsaggouras, 1998) and

a series of social characteristics and standards. The

predominant practice is the lecture given by the

professor to the audience composed of a number of

students (Neumann, 2001). An exception to this

educational model is the integrated approach

followed mainly by the universities of Great Britain,

USA and Australia, which set standards both for the

curricula and the skills development of their

students. It is worth mentioning the Problem Based

Learning-PBL educational model, which is applied

in Aalborg University that focuses on the outcomes

and the student. In fact, new technologies and their

applications have enhanced teaching although even

in the most developed countries they have not

managed to change the prevalent feature of

academic classroom and teaching methods (Raptis

and Rapti, 2006).

The Technical University of Crete runs an e-

learning platform (E-Class), since 2005, which is

applied to almost all higher Greek education

institutions. Almost all tutors upload to E-Class

courseware, course notes, exercises and any useful

information regarding their course. Despite the

encouraging comments and the high degree of

satisfaction expressed by students, another research

on the development of the platform shows that the

system is not fully utilized and is mainly used to

download the notes/lectures of courses. This was the

conclusion following observation by the researchers

380

Evangelia K., F. Nikolaos M. and A. Leonidas Z. (2010).

EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR ON COOPERATIVE LEARNING IN AN ENGINEERING COURSE.

In Proceedings of the 2nd International Conference on Computer Supported Education, pages 380-384

 SciTePress

of this paper, in relation to the self-assessment

digital material prepared for the Decision Support

Systems-DSS course, which is a mandatory course

for third-year students at the Department of

Production Engineering and Management.

The E-Class has a special service (exercises) for

academic staff, which is optionally activated so that

the tutor can prepare exercises in the form of

questions-answers for the students. With regard to

the DSS course, the digital material in the form of

self-assessment exercises was divided into groups of

thematic modules, according to the detailed

curriculum and in the form of multiple-choice or

matching or fill-in-blank questions and answers.

This effort aims to enhance teaching of the course,

familiarize students with the further features of the

platform, improve students‟ knowledge through self-

assessment and support digital interaction for

educational purposes. However, despite the effort

made at the beginning of each semester by the tutors

to instruct all students attending the course in using

the features of E-Class (2-hour presentation and

relevant notes) and despite the effort to develop,

improve, and update the self-assessment material on

an annual basis it becomes evident that students used

it only to download the material of the course and

hardly ever used the self-assessment feature. The

support team of the course noticed that the self-

assessment material is scarcely used, mainly near the

date of examinations, as a source of possible

questions for the examinations.

In the light of these findings we organized a pilot

seminar, in the form of „experiment‟ involving a

group of twenty-seven volunteer undergraduate

students. The main goal was to utilize the digital

material. Therefore, we prepared a seminar based on

cooperative learning that would use different

techniques for the teaching of the same course. This

seminar was held on different days and hours than

the course, and was attended by 27 volunteers. The

idea of the seminar was to integrate the digital

material in the form of 10-minute group exercises

with the active participation of students. It intended

to measure the final performance of participants

compared to non-participants and to evaluate their

preference in five different activities, one of which

was based on the digital material from E-class. In

particular, the five activities combined with effective

speaking/writing (communication skills) and

teamworking skill, were the following: individual

written assignments (at home), oral presentation

with PowerPoint (prepared at home), group research

projects by 2-3 persons (prepared at home during the

semester; a very well-known alternative given as a

project in the context of the course, and followed by

non-participants in the seminar), active participation

in a discussion in class (questions-answers) and E-

Class exercises (10-minute written exercises in

groups of two in class).

The learning objective of the seminar was not

different from the objective specified in the detailed

curriculum of the course. It is noted that at

undergraduate level, effective writing/speaking and

teamworking are some of the transferable skills

suggested by a large number of researchers (Kemp

and Seagraves, 1995; Venetsanopoulos, 2004;

Baldwin, Cahn, Forman, Lehmann and Wischmeyer,

1979; Caroll, Markauskaite and Calvo, 2008). Both

teams of students (seminar/project) had the

advantage of a final oral examination. In particular,

the second team of students consisted of those who

had chosen to prepare a written research project in

groups of 2 or 3 persons instead of participating in

the seminar. Both teams of students who opted to

participate either in the seminar or the project would

be given an overall performance grade based on:

their performance in the laboratory, their

performance at the seminar (27 persons) or their

grade for the project (57 persons) and their

performance during the oral examination on the

course curriculum.

2 SEMINAR DETAILS

The seminar was held in a classroom equipped with

a round table, in order to promote equality and

familiarity and to avoid the sense of „superiority‟ of

the tutor, as is the case when teaching from the desk,

aiming to encourage participation. The seminar

extended over seven sessions of two hours each,

covering Information Technology Systems and

Decisions Theory, within the educational goals of

the course. The tutor assumed also the role of

facilitator of knowledge not authenticity, coordinator

not transmitter of knowledge, who learns through

the students‟ experiences (Felder, 1996). The role of

students changed from a passive receiver to an equal

participant in the building of knowledge, a producer

of knowledge who draws from previous experiences;

they now assumed an active and interactive, not

passive role. In brief, an effort was made to perform

a more student-centered learning process aiming at

the development of students both at knowledge and

at skills level, and less focused on the usual one-

sided knowledge-based approach which is focused

on the principle of transferring knowledge from the

tutor to the student. In addition, we took into account

EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR ON COOPERATIVE LEARNING IN AN

ENGINEERING COURSE

381

that, as set forth by Yorke and Knight (2004), the

learning process is not merely the proof of

knowledge proficiency in specific cognitive objects

yet a more complex process which informally and

tacitly enhances and improves the experiences and

competences of students. For example, it develops,

strengthens and improves experiences and

competences, such as: communication,

teamworking, etc. For all these reasons, although the

development of students is traditionally linked to

their academic progress, the seminar aimed at a

more global development.

The techniques used by the coordinator of the

seminar included the following:

Brief introduction, usually supported by few slides.

Goal: to avoid monologue.

Brainstorming method through words / sentences /

questions written on the board or distributed to the

participants in printed form. Goal: to connect the

previous experiences-knowledge of students to the

educational objective and obtain their active

participation in the class.

Questions-answers method with relevant material

distributed or oral questions made by the

coordinator. Goal: to obtain the active participation

of students, explore pre-existing knowledge, verify

the usefulness of the course, etc.

Students divided in groups of two, by draw. Goal: to

enhance the cooperation of students, working in

groups of two, who do not necessarily know each

other or are friends; indirectly support team work

with persons not picked by the student

himself/herself.

Short (10-minute) group-written exercises in the

class, derived from the self-assessment material of

E-Class. Goal: to enable the cooperation of the

groups of two students in order to achieve the

objective of the exercise and better learn the material

taught.

Presentation of short group-written exercises. The

exercises were prepared in the class by the groups of

two students and then presented to the audience.

Goal: to facilitate the oral expression of knowledge

obtained by the students and improve their oral

skills.

Individual short written assignments to be prepared

at home referring to the topics covered during the

seminar. The assignments were usually delivered at

the next session. Goal: to improve the knowledge of

students, enable deeper knowledge and further

practice. The coordinator evaluated each individual

written assignment in terms of fully elaborated

subject, presentation of the paper, syntax, spelling,

punctuation, and returned it to the student for

feedback and reflection.

Students selected by draw to make an individual

oral presentation of a topic distributed to them.

Usually, presentations were made every 2 weeks.

Goal: to elaborate the topic, prepare the oral

presentation, structure the presentation in a cohesive

and comprehensive manner within the set deadline.

3 RESULTS

In analyzing the results of the pilot initiative we

checked the performance of students and we

delivered a specially formulated questionnaire at the

end of the semester to the participants in the

seminar.

In Fig. 1 we present the performance of both

teams of students on a scale from 0 to 10 (excellent)

[pass grades are from 5 to 10]. We used independent

samples, t-tests, to determine if there were

significant differences in student performance. Only

the performance in the final oral examination was

found to be statistically significant [t (66.33) = 2.12,

p <0.05)]. The effect size, Cohen‟s d, was found to

be 0.46, which is a moderate effect. Furthermore, we

found no statistical differences in performance based

on the students‟ gender.

With regard to the teaching techniques and

methods, we examined their effect in enhancing

effective writing, effective speaking and

teamworking. This research was carried out with a

specially formulated questionnaire using the

Conjoint Analysis (CA) technique. CA (also known

as “trade off analysis”) is a multivariate technique

used to estimate or determine how respondents

develop preferences for products/services. It is

widely used in marketing research and is based on

the premise that consumers assess the value of a

product by combining the separate amounts of value

provided by each attribute (or factor) of the

particular product or service.

In the CA experiment conducted in this study,

the “product” is teaching techniques, which is

decomposed into relevant factors that can be

combined to fully describe them. From the specified

factors and factor levels, hypothetical products

(scenarios) were constructed for students to evaluate.

The 5 specific factors and factor levels (in

parentheses) were as follows: 1. preparation of

individual written assignments at home (every week

or every two weeks), 2. individual presentation of

the assignment using PowerPoint in the class (every

week or every two weeks), 3. preparation of a group

CSEDU 2010 - 2nd International Conference on Computer Supported Education

382

research project by 2-3 persons during the semester

(yes/no), 4. active participation through questions-

answers during the course (yes/no), 5. group written

exercises in class based on digital material (yes/no).

The combination of the five factors and two

levels per factor gave rise to 32 possible scenarios (2

x 2 x 2 x 2 x 2). To reduce the number of profiles to

a manageable size, while at the same time

maintaining randomness, a fractional factorial

design using SPSS was used. This process reduced

the number of profiles to be evaluated to ten (see

Table 1), including two holdout profiles. Holdouts

are scenarios which are rated by the students but are

not used to build the preference model. Instead, they

are used as a check on the internal validity of the

model. The full-profile approach, whereby students

were asked to rate the full descriptions according to

preference, was used. A rating scale anchored from

one to seven, with 1 being “no improvement” and 7

“great improvement” was selected. The study

utilized a verbal description approach. The ten

scenarios used in the questionnaire had the following

form: (scenario 1) „Suppose that every week you

must deliver an individual short written assignment

and every 2 weeks you must prepare an oral

presentation on PowerPoint for the purposes of the

course. You will not be given any group research

projects. In class you will actively participate by

answering questions made by the coordinator for

discussion, and you will prepare together with your

classmates short written exercises that facilitate the

learning process. Do you think that the above

activities improve your written and oral expression

and your ability to work in a team?’ The SPSS

conjoint procedure (SPSS, 2004) was used to

calculate utility scores (or part-worths) for each

individual respondent and for the whole sample.

These utility scores are analogous to regression

coefficients, and are used to estimate the relative

importance of each factor. According to the results

of preference expressed by students, the five factors

rank as follows in terms of importance: 1. short

group exercises in class (E-Class material)

(29.37%); 2. active participation through questions-

answers during the course (26.12%); 3. group

research projects during the semester (21.89%); 4.

preparation of individual written assignments at

home (14.26%), and 5. individual presentation of the

assignment on PowerPoint in class (8.37%).

Our results indicate an adequate level of internal

validity in terms of Pearson‟s product moment

correlation coefficient (r = 0.98, p<0.001), and

Kendall‟s tau (τ = 0.997, p< 0.001) for the whole

sample. With regard to the means used to improve

written-oral expression and cooperation with other

students, there seems to be a preference firstly to

exercises performed in the classroom, based on E-

Class self-assessment digital material, then to

discussion through questions made by the

coordinator, etc.

4 CONCLUSIONS

Although the sample was very small, the initial

research indicated whether tutors can affect the

results of learning by applying a mix of teaching

methods and techniques. The performance in the

final oral examination of the students who

participated in the seminar was higher compared to

the performance of non-participants. It is noted that

the written comments of participants and the high

level of satisfaction expressed by the students

encouraged us to repeat the seminar the following

year and thus achieve higher reliability of our

conclusions. It is worth mentioning that although

students did not generally use the self-assessment

digital material, in the form of 10-minute long

exercises, it was highlighted as the most important

tool to improve the two transferable skills,

communication and teamworking. McDonald also

reached a similar conclusion in 1977 when he

noticed that a significant part of learning was taking

place in group discussions, away from computers yet

regarding computer work.

We do not know whether students would keep the

same order of preference if written group exercises

in class were diversified in connection with the

digital material and had a different content or form.

Prima facie we can conclude that students developed

a better understanding of the usefulness and value of

the self-assessment digital material, for the

additional reason that it promoted the interaction,

exchange and negotiation of ideas both between the

groups of two students and at group level in the

class. Additional conclusions came to light: it

became evident that the theoretical part of an

academic course can become more „exciting‟ and

interesting to students if a mix of teaching

techniques is used. In this context, numerous tools,

techniques and methods can be applied, and any

digital means can, directly or indirectly, become part

of everyday practice and be useful.

EVALUATING THE EFFECTIVENESS OF A PILOT SEMINAR ON COOPERATIVE LEARNING IN AN

ENGINEERING COURSE

383

Figure 1: Comparing the performance of student groups.

Table 1: Scenarios through Fractional Factorial Analysis.

s/n

Individual written

assignments

Oral presentation using

PowerPoint

Group research project

during the semester

Discussion through

questions-answers

Short exercises in

groups of two

Status

Every week

Every 2 weeks

Yes

Holdout

Every 2 weeks

Yes

Design

Every week

Yes

Design

Every 2 weeks

Yes

Design

Every week

Yes

Design

Every 2 weeks

Yes

Holdout

Every week

Every 2 weeks

Yes

Design

Every 2 weeks

Every week

Yes

Design

Every week

Every 2 weeks

Yes

Design

Every 2 weeks

Every week

Yes

Design

REFERENCES

Matsagouras, Η., 1998. Strategies of teaching. The critical

thought in the teaching practice. Ed. Gutenberg (in

Greek), 2

edition, Athens.

Raptis, Α., Α. Rapti, 2006. Learning and teaching during

the information technology society. Vol. Α‟. Ed.

Raptis (in Greek), Athens.

Baldwin, J.C., C.R. Cahn, J.W. Forman, H. Lehmann,

C.R. Wischmeyer, 1979. A Model Undergraduate

Electrical Engineering Curriculum. IEEE Transaction

on Education, vol. E-22(2).

Caroll, L.N., L. Markauskaite and R. A. Calvo, 2008. E-

Portfolios for developing tranferable Skills in a

Freshman Engineering Course. IEEE Transaction on

Education, 50(4).

Felder, R.M., 1996. Matters of style. ASEE Prism, 6(4),

18-32.

Kemp, J. and L. Seagraves, 1995. Transferable Skills-can

higher education deliver? Studies in Higher Education,

20(3), 315-328.

McDonald, B., 1977. The educational evaluation of

NDPCAL, British Journal of Educational Technology,

8(3).

Neumann, R., 2001. Disciplinary differences and

university teaching. Studies in Higher Education,

26(2), 135-146.

Venetsanopoulos, A., 2004. Advice to Young Engineers,

IEEE Signal Processing Magazine, vol. July.

7.9

6.4

7.3

7.8

5.7

Average laboratory work grade

Average seminar/project

grade

Average Oral Grade

Average Overall Performance

Students with project

Seminar students

CSEDU 2010 - 2nd International Conference on Computer Supported Education

384