UNDERGRADUATION COURSE EVALUATION
A Case of Success
Regina Lúcia de Oliveira Moraes, José Geraldo Pena de Andrade and Marcelo Gonçalves Diotto
Superior Center of Technological Education - CESET, University of Campinas - UNICAMP, Campinas, Brazil
Keywords: Undergraduate Program Evaluation, Web-based Undergraduate Program Evaluation Software.
Abstract: One important question in education management is the best way to conduct institutional evaluation. This
relevant process that helps the institution in its self-knowledge is essential to verify if the institution goals
are being achieved and to plan quality improvements. One important matter to be considered is the time
between the forms filling by the stakeholders and the reports of the results. If this time is too long the results
has a minor impact in the actions that will bring benefits to people that are involved in the process. To deal
with this matter a software support is mandatory. This work presents a effort of the institution to establish an
evaluation process supported by web-based software. The process was extensively discussed among
students, lecturers, coordinators and principals in order to create forms and questions that best fit for each
type of stakeholder involved in the process. This work presents the on-line indicators of the institution that
are being used to speed up management actions.
1 INTRODUCTION
Evaluation, which is understood as a continuous and
systematic process, allows people to identify if the
goals of the institution under evaluation are being
achieved successfully. In this process, their strengths
and weaknesses are revealed, as well as the elements
to sustain discussions that may result in guidelines
for possible revisions in its design (Vieira Neto,
2003). The term process is considered as a set of
actions (Freitas e Arica, 2008). In the scope of this
work, these actions are developed in order to
evaluate students learning and infrastructure
conditions.
Evaluation, in general, is essentially a complex
phenomenon and can not be adequately understood
if we do not observe the results with many eyes and
do not have support from analysis tools (Dias
Sobrinho, 2000).
Among the key components of an evaluation, the
methodologies for the data collection, data analysis
and data interpretation must be established, so that
evaluation purposes are achieved in the best way
possible. The inadequate choice of these
methodologies may render the evaluation unviable
or ineffective.
Data collection generally occurs through
interviews and/or questionnaires with the institution
representative people involved in the evaluation
process. According to Dey and Fenty (Dey and
Fenty, 1999), evaluations with such approach are of
a quantitative nature, selecting the indicators in
advance.
The indicators of the evaluation process
constitute a major propellant for institution changes.
The evaluation of the institution gives us at the same
time a situation diagnosis and provides indicators
that can be valuable as a reference to prepare action
plans, strengthening or correcting routes to improve
the quality of offered education.
Often information from an evaluation process is
explored through descriptive statistics, producing
extensive reports that, in most cases, do not allow
interested people make the best use of the
information (Vieira Neto, 2003). Also, the lack of
computational tools to support and consolidate
collected data makes the process infeasible.
The main goal of this work is to present an
experience developed and implemented of an
undergraduate academic program evaluation process
supported by software. The process takes into
account the evaluation of courses modules and the
institution infrastructure by students and the
evaluation of students by responsible lecturers.
The remainder of this paper is organized as
follows: the next section presents the undergraduate
343
de Oliveira Moraes R., Pena de Andrade J. and Gonçalves Diotto M. (2009).
UNDERGRADUATION COURSE EVALUATION - A Case of Success.
In Proceedings of the First International Conference on Computer Supported Education, pages 343-346
DOI: 10.5220/0001971203430346
Copyright
c
 SciTePress

program evaluation details; Section 3 describes the
software product developed; results are presented in
Section 4; finally Section 5 concludes the paper.
2 THE EVALUATION
It is believed that the evaluation process should be
considered based on two main and essential
elements in an educational institution: the lecturers
and students. For the lecturers usually is used as
criteria: professional qualifications, professional
experience and scientific production. Undoubtedly
they are important, but not necessarily ensure the
quality of teaching, while other criteria no less
important for the educational quality, such as
didactics, the way used to provide doubts
clarification and punctuality, are not always present
in evaluations in general.
The Undergraduate Program Evaluation at
UNICAMP was bringing a disincentive to students
to participate in the process. Moreover, the model
did not provide any way of evaluating the students
by the lecturers.
Done on paper, read by optical readers, stored in
database and later consolidated, the evaluation was a
slow process and the results were released with a
gap of nearly six months.
This fact made us think that it is time to use the
computing resources we had available at the
institution to support the evaluation process
modernization so that students and lecturers could
help in improving the courses, the institution social
life and its infrastructure.
The Undergraduate Program Evaluation
presented in this work is an interactive process that
was developed in cooperation with all members of
the Institution. It was consensus that the main
objectives of the evaluation should be centered on
the use of the evaluation results to maintain and
improve the quality and relevance of our program.
In this context, the evaluation system presents an
assessment tool taking into account the students and
lecturers perception.
Another important conclusion was a consensus
that lecturers should evaluate students’ behavior
during classes and express themselves about
students’ commitment. Also, the infrastructure
evaluation should be done in order to provide
knowledge about the improvements that are
necessary. The student and lecturers participation is
completely optional and spontaneous.
2.1 System Requirements
Based on the results of the discussion, the
Undergraduate Program Evaluation should be
composed by questionnaires with pre-established
questions but also open questions to enable students
and lecturers to submit free opinions about every
topic related to the institution. These open questions
also help the improvement of pre-established
questions for future versions.
The evaluation results should be monitored in
real-time allowing course coordinators and
principals to take quick actions if a critical problem
is detected. It is important that participants can
evaluate their issues remotely using internet access.
Moreover, the number of items and the time it takes
to complete the questionnaire are important. So, the
questionnaire should be composed by few pre-
established questions. When the participants leave
the system by any reason, they can continue later at
the point where they stopped. The necessary
information about students and the modules in which
they are enrolled must be obtained from the central
system of the university.
This version of the evaluation system should be
composed by four questionnaires;
a) students’ satisfaction about the undergraduate
course and the institution. We want to know if the
course they are attending was its preferred option
and why they chose the course.
b) students’ perception about the institution
infrastructure and the access of the student to
administrative structure.
c) Courses’ evaluation by the students to evaluate
the objective of the courses, didactic and
organization of the lecturers, teaching materials, the
relevance of the subjects addressed in tests, lecturers
motivation and students self-assessment
d) Class’ evaluation by the lecturer. It’s possible to
analyze the class by the viewpoint of the lecturer in
same aspects, as: objective of the courses, students’
commitment and motivation, the bibliography
availability and the background of the students.
The software was developed based on these
requirements. The next section presents technical
details of software and considerations about the
evolution of the evaluation system.
3 THE SOFTWARE
The software architecture was totally based on open
source software, which minimizes the costs of the
development. The chosen operating system was
CSEDU 2009 - International Conference on Computer Supported Education
344

Debian Linux (Debian, 2008) and the application
server is an Apache HTTP Server (Apache, 2008)
with PHP Scripting Language (PHP, 2008) installed
on it; all the data of the software is stored in a
MySQL Database (MySQL, 2008), including text
forms and written questions.
When a user accesses the system the first
interface that he observes is the status of the
evaluation that should be completed, which shows
the forms that are already completed (“Done”) and
pending (“Questions Pending”). Figure 1 shows the
interface of the lecturer that presents all the subject
that a specific lecturer have under his responsibility.
Figure 1: Lecturer Status Interface.
The system is designed to work with three
different users: students, lecturers and
coordinators/principals. Student are asked to fill in
the evaluation forms of the subjects he is enrolled.
The evaluation contents are exemplified in
Figure 2, that partially shows the form of a specific
subject been evaluated by the student.
Figure 2: Example of the Evaluation’ Contents.
The student and the lectures can follow the
evaluations done in a real time very easily. The
lecturers have access to their subjects and students to
their courses. Figure 3 shows partially the report that
lecturers have access.
The coordinator/principal has permission to
verify the assessment reports of all disciplines of the
organization and to monitor how the progress of
filling in forms by students and lecturers are been
done.
Figure 3: Example of Lecturers/Directors Report.
It is important to notice that each time the
participant enters an answer or comment the charts
are updated and new comments are inserted.
4 RESULTS OBTAINED
The evaluation process were applied four times. The
first and second time that it was applied there was no
significant participation neither by the students nor
by the lecturers, but it was enough to ensure that
major adjustments could be done both in the
evaluation process and supported software.
At that time, evaluation results were released
two months after the end of the process. Although
this period is much better than the manual process,
the students complained about information delay.
This fact can coroborate to not encourage the
students to participate in the process the second
time. Now, with the new version of the system, the
results are released instantaneously and people are
able to accompany them daily.
Table 1 and Figure 4 show the participation of
the students and lectures. It is possible to observe
that in the third time the evaluation was applied,
92% of lecturers and 42% of students participated in
the process. These results are due to the internal
market of the importance of the evaluation and a
virtual reminder that appears every time the users
logged in the institution’s website. Also, it is good to
mention that the participation in the forth time
decreased. Lecturers participation decreased 18%
and students participation 9,3%. The reason why this
undesired event occurred was being investigated at
the time this work was written. But clearly we can
see that the lecturer participation encourages the
student participation.
UNDERGRADUATION COURSE EVALUATION - A Case of Success
345

Table 1: Students and Lecturers Participation.
1S/07 2S/07 1S/08 2s/08
Students 20,3% 11,7% 41,9% 30,2%
Lecturers 47% 34% 92% 57%
It is believed that the spontaneous participation
should be kept, but the institution intends to create a
way to ensure that each student is notified about the
process.
Figure 5 shows another significant result. The
students prefer to participate without identification.
Maybe they are concerned about any type of
retaliation due to their criticism and comments.
The students’ comments about installations and
critical problems in a specific subject allows the
administrators to act before the end of the evaluation
process.
20,3
11,7
41,9
32,6
47,0
34,0
92,0
74,0
0
10
20
30
40
50
60
70
80
90
100
1s/2007 2s/2007 1s/2008 2s/2008
students' participation
lecturers' participation
Figure 4: Students and Lecturers participation.
9,65
5,87
18,55
13,03
10,53
5,78
22,66
18,77
0
5
10
15
20
25
30
1s/2007 2s/2007 1s/2008 2s/2008
answ er the avaliation w ith identification
answ er the avaliation w ithout identification
Figure 5: Students with and without identification.
5 CONCLUSIONS
The evaluation process of undergraduation subjects,
lecturers and the infrastructure of the institution
contributes to the improvement of education quality.
At this point of view, this work presents a new
evaluation process that was applied four times and
has been contributed positively to the improvement
of the institution and courses.
The use of computer resources aims to stimulate
the participation of the people involved in the
process, facilitating the filling of the and helping to
obtain and disseminate the results.
Due to the programming structure of the system,
the process coordinators can easily include or
exclude new texts and / or questions in the forms,
since these information are stored in database
management systems and could be easily
manipulated.
Finally, the evaluation system enables
educational institution managers to monitor in real
time the progress of evaluation, allowing implement
corrective actions during and after the process.
ACKNOWLEDGEMENTS
The authors thanks all CESET/UNICAMP lecturers
and students who help us to define the evaluation of
the undergraduation course and also for their
participation in the evaluation process.
REFERENCES
Apache, 2008. Apache HTTP Server. Available in
http://www.apache.org. Last access December 2008.
Debian, 2008. Linux Operating System. Available in
http://www.debian.org. Last access December 2008.
Dey, E. L. e Fenty, J. M., 1999. Avaliação em educação
superior: técnicas e instrumentos. In: Sousa, E. C. B.
M. (Coord.). Técnicas e instrumentos de avaliação.
2ªed., Brasília:Universidade de Brasília, 1999, p. 1-35.
Dias Sobrinho, J., 2000. Avaliação da Educação Superior.
Petrópolis, Vozes.
Freitas, A.L.P. e Arica, G.M., 2008. A auto-avaliação de
IES: um modelo para avaliação das disciplinas
curriculares segundo a percepção do corpo discente.
Revista Iberoam. de Educación, no 44/7, enero/2008.
ISSN 1681-5653.
MySQL, 2008. MySQL Database. Available in
http://www.mysql.com. Last access December 2008.
Perrenoud, P., 1999. Avaliação: da excelência à regulação
das aprendizagens. Artmed, 1999, Porto Alegre-Brasil.
PHP, 2008. PHP Scripting Language. Available in
http://www.php.net. Last access December 2008.
Vieira Neto, J., 2003. Uma abordagem qualitativa em
avaliação de Instituições de Ensino Superior. Revista
da Rede de Avaliação Institucional da Educação
Superior, Ano 8, Vol. 8, nº 3, ISSN 1414-4077.
CSEDU 2009 - International Conference on Computer Supported Education
346